Custom Detection Rules FAQ

When you are starting to write custom detection rules, it's expected that you will have questions. Read on to understand more about using custom detection rules in your organization.

How many custom detection rules can I create?

By default, InsightIDR limits the creation of custom detection rules to 50 per organization. Rules that have been turned off do not count towards the number of custom detection rules.

If your organization requires more than 50 custom detection rules, you can contact your Customer Success Manager to request a limit adjustment.

What LEQL capabilities can I use to write a custom detection rule?

You can use the same LEQL components to write the logic for your custom detection rule as you would use to build queries in Log Search. This includes the ability to reference variables in a query, for example where (asset in ${pci_assets}).

However, there are some limitations. At this time, keyword search is not supported for detection rule logic. This means that you are unable to query data across one key, for example where (evil.exe).

Use operators and explicit AND statements in your LEQL query

Custom detection rules require the use of operators and explicit AND statements to remove ambiguity within the rule logic. For example, you cannot enter the query where("result" = "FAILED_BAD_PASSWORD" "geoip_country_name" != "United States")

Instead, you must use the AND operator between the keys FAILED_BAD_PASSWORD and geoip_country_name: where("result" = "FAILED_BAD_PASSWORD" AND "geoip_country_name" != "United States")

What is the difference between a custom detection rule and a basic detection rule?

In general, custom detection rules provide more robust customization options that allow you to detect specific activity ingested through InsightIDR’s detection system. However, there are some cases where you should create a basic detection rule instead.

Compare custom detection rules to basic detection rules to determine which feature best fits your use case.

CapabilityCustom detection rulesBasic detection rules
Detection thresholds and conditional logicSupportedNot supported
Creation of exceptions to rulesSupportedNot supported
Investigation triage capabilities:
- Ability to group detections with open investigations
- Attribution of actors from matching events
- Integrated view of rule logic in investigations evidence
SupportedNot supported
Modification historySupportedNot supported
Ability to set rule action to create investigations, generate notable events, or turn off the ruleSupportedNot supported
Ability to set rule priority on generated investigationsSupportedSupported
Alerting on key-value pair-based detectionsSupportedSupported
Connection to automation workflows with InsightConnectSupportedSupported
Notifications through SlackNot supportedSupported
Alerting on inactivity and change detectionsNot supportedSupported
Alerting on logs that are sent directly to Log Search through the logging.json configuration method or through the Log Search APINot supportedSupported
What data can I create a custom detection rule for?

You can write custom detection rules on logs that come from event sources through the Collector, from the Insight Network Sensor, and from the Insight Agent.

Data that you cannot create custom detection rules for

You cannot write rules on logs sent to InsightIDR through the logging.json configuration method because these logs are sent directly to Log Search, bypassing our detection rule engine. To detect on events in these logs, create a basic detection rule in Log Search.

How can I determine what data my custom detection rule detects on?

Custom detection rules are scoped by event type. You must select an event type when creating your rule to define the data to detect on. If you want to further refine the data set, you can also select specific logs and log sets that map to your chosen event type. Read more about how log sets map to event types.

How do I build rules for raw logs and unparsed data?

The Raw Logs and Unparsed Data log sets do not have dedicated schemas. To find the keys and values for these log sets, you can navigate to Log Search in the InsightIDR left menu and run a blank query. The results will display all available key-value pairs. You can use this information to write your LEQL query and add conditions.

How can I test my rule to make sure it isn’t noisy?

There are two ways to test your custom rule: evaluate your query in Log Search, and use the Assess Activity rule action.

You can evaluate your query in the rule creation modal to view the data that is returned in Log Search. To do so, click the Evaluate in Log Search button when writing your query in the Custom Detection Rules modal.

Use the Assess Activity rule action

You can also use the Assess Activity Rule Action to test your rule after it has been saved. To do so, select Assess Activity from the Rule Action dropdown on the Rule Details peek panel.

Assess Activity tracks the number of detections that occur over a 7-day period and measures the relative activity score for the rule. After 7 days, the Rule Action will automatically be switched Off, unless you manually change it. You can use this option to preview detection activity for your rule to ensure it won’t be too noisy in your environment.

How do I exclude specific users or assets from my rule?

There are a couple of ways you can exclude specific users, assets, and IP addresses, from your rule:

  • You can use a not clause in the rule logic to specify what you don’t want the rule to detect on. For example, if you would like to detect on successful logins from outside of the United States or United Kingdom, you could enter the query: where("result" = "SUCCESS" AND "geoip_country_name" not contains-any["United Kingdom","United States"])
  • You can also add exceptions to your rule after it is created. Exceptions allow you to modify the rule action for specific key-value pairs to either suppress detections or increase the action or priority.
How do I set up notifications for my custom detection rules?

You can configure email notifications for investigations created by custom detection rules. You can also set up automation workflows through InsightConnect to notify you when a detection occurs. To learn about how to add automation, read Get started with ABA Automation.

How do the alerts generated by my rule get grouped in investigations?

Alerts generated by custom detection rules are grouped into investigations based on the rule conditions that you set:

  • If you specify one or more grouping keys and a threshold, alerts that are generated from this rule will be grouped into investigations according to the grouping keys.
  • If you only specify a threshold, alerts that are generated from this rule will be grouped together in the same investigation.
  • If you do not specify any conditions, investigations will follow the same grouping as InsightIDR’s attacker behavior analytics (ABA) detection rules. If a detection occurs for a user or asset with an open investigation created from this rule, the related alert is added to the open investigation. If there is not an existing investigation, a new investigation is automatically created.