Custom Detection Rules FAQ

By default, InsightIDR limits the creation of custom detection rules to 50 per organization. Rules that have been turned off do not count towards the number of custom detection rules.

If your organization requires more than 50 custom detection rules, you can contact your Customer Success Manager to request a limit adjustment.

You can use the same LEQL components to write the logic for your custom detection rule as you would use to build queries in Log Search. This includes the ability to reference variables in a query, for example where (asset in ${pci_assets}).

However, there are some limitations. At this time, keyword search is not supported for detection rule logic. This means that you are unable to query data across one key, for example where (evil.exe).

Use operators and explicit AND statements in your LEQL query

Custom detection rules require the use of operators and explicit AND statements to remove ambiguity within the rule logic. For example, you cannot enter the query where("result" = "FAILED_BAD_PASSWORD" "geoip_country_name" != "United States")

Instead, you must use the AND operator between the keys FAILED_BAD_PASSWORD and geoip_country_name: where("result" = "FAILED_BAD_PASSWORD" AND "geoip_country_name" != "United States")

In general, custom detection rules provide more robust customization options that allow you to detect specific activity ingested through InsightIDR’s detection system. However, there are some cases where you should create a basic detection rule instead.

Compare custom detection rules to basic detection rules to determine which feature best fits your use case.

You can write custom detection rules on logs that come from event sources through the Collector, from the Insight Network Sensor, and from the Insight Agent.

Data that you cannot create custom detection rules for

You cannot write rules on logs sent to InsightIDR through the logging.json configuration method because these logs are sent directly to Log Search, bypassing our detection rule engine. To detect on events in these logs, create a basic detection rule in Log Search.

Custom detection rules are scoped by event type. You must select an event type when creating your rule to define the data to detect on. If you want to further refine the data set, you can also select specific logs and log sets that map to your chosen event type. Read more about how log sets map to event types.

The Raw Logs and Unparsed Data log sets do not have dedicated schemas. To find the keys and values for these log sets, you can navigate to Log Search in the InsightIDR left menu and run a blank query. The results will display all available key-value pairs. You can use this information to write your LEQL query and add conditions.

There are two ways to test your custom rule: evaluate your query in Log Search, and use the Assess Activity rule action.

Evaluate your query in Log Search

You can evaluate your query in the rule creation modal to view the data that is returned in Log Search. To do so, click the Evaluate in Log Search button when writing your query in the Custom Detection Rules modal.

Use the Assess Activity rule action

You can also use the Assess Activity Rule Action to test your rule after it has been saved. To do so, select Assess Activity from the Rule Action dropdown on the rule details peek panel.

Assess Activity tracks the number of detections that occur over a 7-day period and measures the relative activity score for the rule. After 7 days, the Rule Action will automatically be switched Off, unless you manually change it. You can use this option to preview detection activity for your rule to ensure it won’t be too noisy in your environment.

There are a couple of ways you can exclude specific users, assets, and IP addresses, from your rule:

• You can use a not clause in the rule logic to specify what you don’t want the rule to detect on. For example, if you would like to detect on successful logins from outside of the United States or United Kingdom, you could enter the query: where("result" = "SUCCESS" AND "geoip_country_name" not contains-any["United Kingdom","United States"])
• You can also add exceptions to your rule after it is created. Exceptions allow you to modify the rule action for specific key-value pairs to either suppress detections or increase the action or priority.

You can configure email notifications for investigations created by custom detection rules. You can also set up automation workflows through InsightConnect to notify you when a detection occurs. To learn about how to add automation, read Get started with ABA Automation.

Alerts generated by custom detection rules are grouped into investigations based on the rule conditions that you set:

• If you specify one or more grouping keys and a threshold, alerts that are generated from this rule will be grouped into investigations according to the grouping keys.
• If you only specify a threshold, alerts that are generated from this rule will be grouped together in the same investigation.
• If you do not specify any conditions, investigations will follow the same grouping as InsightIDR’s attacker behavior analytics (ABA) detection rules. If a detection occurs for a user or asset with an open investigation created from this rule, the related alert is added to the open investigation. If there is not an existing investigation, a new investigation is automatically created.