Advanced Event Source Settings

Multi-Domain Environments

If you have multiple domains in your environment, it is important that you specify a default domain for all event sources. This setting ensures that SIEM (InsightIDR) knows which domain should be used to attribute users to, particularly when that data is not provided in the event log. A default domain can be set for all event sources when configuring user attribution settings. To use a different domain for a specific event source, you can customize it directly in the event source’s configuration settings.

Rapid7 Encryption Certificate

When using TCP to send event source data by syslog, you can also choose to encrypt that data. When configuring an event source, choose TCP under collection methods and select the “Download Certificate” button. The certification is called Rapid7CA.pem and will allow SIEM (InsightIDR) and the event source to “trust” each other during log forwarding.

Rapid7 Recommends importing the certificate file on the same machine as the vendor or application you are connecting to SIEM (InsightIDR) as an event source. Use your administrative tool or vendor in order for your machine to ingest the certificate.

The certificate file contains two keys: a public key for your organization, and the Rapid7 key, which created the org-key.

Inactivity Timeout Threshold

This setting applies only to DHCP and VPN event sources. The inactivity Timeout Threshold setting allows you to specify in minutes how long an event source should be inactive before it enters an error state.

Active Failover Partner

If you have two DHCP servers configured in an active/passive relationship, you can specify the active partner.

Unparsed Logs

Learn about how unparsed logs affect your event source.

Attribution Source

In SIEM (InsightIDR), attribution refers to the attempts the system makes to identify which assets, accounts, and users are involved in the collected log activity. For example, when an event log states that the activity was performed by the account jdoe, SIEM (InsightIDR) uses previously collected information from other event sources to determine whether that account is associated with the user Jane Doe or John Doe.

Attribution source can be set for all event sources when configuring user attribution settings. To use a different attribution source for a specific event source, you can customize it directly in the event source’s configuration settings.

Filter Response Data

ℹ️

New Feature

This feature is currently in Early Access (EA) and is available only for the Microsoft Security cloud event source. Support for additional cloud event sources will be introduced in a future release.

For event sources that receive high volumes of data, you can add a filter to capture only the specific items you need.

This filter setting is optional and is only available for cloud collection method event sources. If you leave this field blank, all data is collected without filtering.

⚠️

Filter processing order and non-matching items

The filter is applied after the event source has collected all of the data. Non-matching results are removed after retrieval.
If you use the filter to include only specific items, you can’t retrieve items that don’t match. The event source doesn’t store or forward items that don’t meet the filter criteria.

Add a Filter

To add a filter to a new event source:

From the Command Platform main menu, go to Data Connectors > Data Collectors.
Go to the Event Sources tab, then click Add Event Source.
Configure the event source as required.
In the Alert filter field, enter a JSON filter configuration to specify which items to include. Only items that match your filter criteria will be included in the data collected by the event source.
Click Save.

To add a filter to an existing event source:

From the Command Platform main menu, go to Data Connectors > Data Collectors.
Go to the Event Sources tab.
Find the event source you want to update, and click Edit. This opens the configuration panel.
In the Alert filter field, enter a JSON filter configuration to specify which items to include. Only items that match your filter criteria will be included in the data collected by the event source.
Click Save.

Filter Configuration Format

Your filter configuration should be a valid JSON string following one of these formats:

Single Filter Rule


{"value": ["high"], "fields": ["severity"]}

Multiple Filter Rules


[
  {"value": ["high"], "fields": ["severity"]},
  {"value":["@example.com"],"fields":["userPrincipalName"]}
]

Filter Rule Structure

Each filter rule must contain:

value (required): A list of strings to search for in the response data
fields (optional): A list of field paths to search in, or omit/set to null to search all fields

Best Practices

Follow these best practices when you create your filter configuration:

Leave the filter empty to collect all data. If you don’t provide any filter configuration and the field contains an empty string, all items are collected without filtering.
Substring matching is used. A filter value will match if it appears anywhere within a field value. For example, "example.com" will match "user@example.com" or "admin@example.com".
Nested fields are supported. Use dot notation to access nested fields (for example, "user.email"), and the filter will recursively extract all string values from that location.
Use specific field paths for better performance. When you know which fields to filter on, specify them explicitly (for example, ["severity"], ["user.email"]) rather than searching all fields.
The filter uses OR logic. If you specify multiple filter rules, an item will be included if it matches any of the rules.
Matching is case-insensitive. Your filter values will match regardless of case. For example, "high" will match "HIGH", "High", or "high".

Filter Examples

Example 1: Filter by severity

To include only high and critical severity alerts, use this JSON format:


{"value": ["high", "critical"], "fields": ["severity"]}

This filter includes only the alerts where the Severity field contains “high” or “critical”.

Example 2: Filter by user email

To include only related to a specific user, use this JSON format:


{"value": ["user@example.com"], "fields": ["userPrincipalName"]}

In this example, only alerts where the userPrincipalName field contains “user@example.com” will be included.

Example 3: Filter by multiple criteria

Include alerts matching multiple different criteria:


[
  {"value": ["user@example.com"], "fields": ["userPrincipalName"]},
  {"value": ["critical"], "fields": ["severity"]}
]

This will include alerts if they contain “user@example.com” in the userPrincipalName field OR if they contain “critical” in the severity field.

Example 4: Search across all fields

Include only items containing “malware” anywhere in their data:


{"value": ["malware"], "fields": null}

This searches the entire data structure for the string “malware”.

Example 5: Filter by nested field

Include only items based on a nested field path:


{"value": ["suspicious"], "fields": ["category.subCategory"]}

This will navigate to the nested field and include only items with matching values found there.

How Field Matching Works

When you specify a field path, the filter navigates to that location in the data and extracts all string values within that structure, including any nested fields.

Simple string values are extracted and matched directly
Objects have all their string values recursively extracted
Arrays have all their string values recursively extracted
Nested structures are fully traversed to find all strings

Response Filtering Reference

The filtering system allows you to include only specific items from collected data based on field values and matching criteria. Filters support case-sensitive matching, multiple match types, and logical combinations.

Basic Filter Structure

A basic filter consists of:


{
  "value": ["string1", "string2"],
  "fields": ["fieldName"],
  "case_sensitive": false,
  "match_type": "contains"
}

Filter Parameters

Parameter	Type	Required	Default	Description
`value`	Array of strings	Yes	-	Values to match against
`fields`	Array of strings or null	Yes	-	Field paths to search. Use `null` to search all fields.
`case_sensitive`	Boolean	No	`false`	Whether matching should be case-sensitive.
`match_type`	String	No	`"contains"`	How to match values (see Match Types below).

Match Types

Match Type	Description	Example
`contains`	Field contains the value anywhere (default)	`"malware"` matches `"detected malware activity"`
`prefix`	Field starts with the value	`"admin"` matches `"admin@example.com"` but not `"user@admin.com"`
`suffix`	Field ends with the value	`"@example.com"` matches `"user@example.com"` but not `"example.com.au"`
`full_match`	Field exactly equals the value	`"critical"` matches only `"critical"`, not `"critically"`

Logical Operators

`any_of` (OR Logic)

Include items that match at least one of the filters:


{
  "any_of": [
    {"value": ["high", "critical"], "fields": ["severity"]},
    {"value": ["admin"], "fields": ["role"]}
  ]
}

This includes items that have high or critical severity OR admin role.

`all_of` (AND Logic)

Include items that match all of the filters:


{
  "all_of": [
    {"value": ["high", "critical"], "fields": ["severity"]},
    {"value": ["failed"], "fields": ["status"]}
  ]
}

This includes items that have high or critical severity AND failed status.

Nesting Operators

Operators can be nested for complex logic, up to a maximum depth of 5 levels:


{
  "all_of": [
    {"value": ["production"], "fields": ["environment"]},
    {
      "any_of": [
        {"value": ["high", "critical"], "fields": ["severity"]},
        {"value": ["malware"], "fields": null}
      ]
    }
  ]
}

This includes production items that have (high or critical severity OR contain "malware" anywhere).

⚠️

Nesting Depth Limit

Filter configurations can be nested up to 5 levels deep. Exceeding this limit will result in a validation error.

Filter Examples

Example 1: Filter by severity (case-insensitive)

Include only high and critical severity alerts:


{
  "value": ["high", "critical"],
  "fields": ["severity"]
}

This matches "High", "HIGH", "critical", "Critical", and so on.

Example 2: Filter by user email (case-sensitive)

Include only alerts for a specific user with exact case matching:


{
  "value": ["user@example.com"],
  "fields": ["userPrincipalName"],
  "case_sensitive": true
}

This matches only "user@example.com", not "User@example.com".

Example 3: Filter by email domain (suffix match)

Include only items from a specific domain:


{
  "value": ["@company.com"],
  "fields": ["email"],
  "match_type": "suffix"
}

This matches "admin@company.com", "user@company.com", and so on.

Example 4: Filter by user prefix

Include only admin users:


{
  "value": ["admin_"],
  "fields": ["username"],
  "match_type": "prefix",
  "case_sensitive": false
}

This matches "admin_john", "Admin_Jane", "ADMIN_DOE", and so on.

Example 5: Exact match only

Include only items with exact status value:


{
  "value": ["ACTIVE"],
  "fields": ["status"],
  "match_type": "full_match",
  "case_sensitive": true
}

This matches only "ACTIVE", not "ACTIVE_USER" or "active".

Example 6: Search across all fields

Include items containing “malware” anywhere in their data:


{
  "value": ["malware"],
  "fields": null
}

This searches the entire data structure recursively.

Example 7: Multiple criteria with OR logic

Include alerts matching multiple different criteria:


{
  "any_of": [
    {"value": ["user@example.com"], "fields": ["userPrincipalName"]},
    {"value": ["critical"], "fields": ["severity"]}
  ]
}

This includes alerts with "user@example.com" OR "critical" severity.

Example 8: Multiple criteria with AND logic

Include only failed login attempts from admin users:


{
  "all_of": [
    {"value": ["admin"], "fields": ["role"], "match_type": "prefix"},
    {"value": ["failed"], "fields": ["status"]},
    {"value": ["login"], "fields": ["eventType"]}
  ]
}

All three conditions must be met for inclusion.

Example 9: Complex nested logic

Include production critical alerts or any security-related items:


{
  "any_of": [
    {
      "all_of": [
        {"value": ["production"], "fields": ["environment"]},
        {"value": ["critical"], "fields": ["severity"]}
      ]
    },
    {
      "value": ["security", "malware", "breach"],
      "fields": ["category"]
    }
  ]
}

This matches: (production AND critical) OR (security/malware/breach category).

Example 10: Filter by nested field

Include items based on a nested field path:


{
  "value": ["suspicious"],
  "fields": ["category.subCategory"]
}

This navigates to the nested field and includes matching items.

Example 11: Case-sensitive domain filtering

Include only items from specific case-sensitive domains:


{
  "value": ["@CompanyName.com"],
  "fields": ["email"],
  "match_type": "suffix",
  "case_sensitive": true
}

This matches "user@CompanyName.com" but not "user@companyname.com".

How Field Matching Works

When you specify a field path, the filter navigates to that location in the data and extracts all string values within that structure, including any nested fields.

Simple string values are extracted and matched directly.
Objects have all their string values recursively extracted.
Arrays have all their string values recursively extracted .
Nested structures are fully traversed to find all strings.
Field paths use dot notation for nested fields (for example, "user.email").

Legacy Format Support

The original array-based format is still supported for backward compatibility:


[
  {"value": ["user@example.com"], "fields": ["userPrincipalName"]},
  {"value": ["critical"], "fields": ["severity"]}
]

This is equivalent to using any_of:


{
  "any_of": [
    {"value": ["user@example.com"], "fields": ["userPrincipalName"]},
    {"value": ["critical"], "fields": ["severity"]}
  ]
}

Best Practices

Use case_sensitive: false (default) for user input to avoid case-related issues.
Use match_type: "full_match" when you need exact values, such as status codes.
Use match_type: "prefix" for hierarchical identifiers, such as usernames.
Use match_type: "suffix" for domain filtering.
Use fields: null sparingly as it searches all fields and may result in a slower performance.
Prefer any_of over legacy array format for clarity.
Use all_of for security filtering where multiple conditions must be met.
Nest operators for complex business logic requirements.

Advanced Event Source Settings

Multi-Domain Environments

Rapid7 Encryption Certificate

Inactivity Timeout Threshold

Active Failover Partner

Unparsed Logs

Attribution Source

Filter Response Data

Add a Filter

Filter Configuration Format

Single Filter Rule

Multiple Filter Rules

Filter Rule Structure

Best Practices

Filter Examples

Example 1: Filter by severity

Example 2: Filter by user email

Example 3: Filter by multiple criteria

Example 4: Search across all fields

Example 5: Filter by nested field

How Field Matching Works

Response Filtering Reference

Basic Filter Structure

Filter Parameters

Match Types

Logical Operators

any_of (OR Logic)

all_of (AND Logic)

Nesting Operators

Filter Examples

Example 1: Filter by severity (case-insensitive)

Example 2: Filter by user email (case-sensitive)

Example 3: Filter by email domain (suffix match)

Example 4: Filter by user prefix

Example 5: Exact match only

Example 6: Search across all fields

Example 7: Multiple criteria with OR logic

Example 8: Multiple criteria with AND logic

Example 9: Complex nested logic

Example 10: Filter by nested field

Example 11: Case-sensitive domain filtering

How Field Matching Works

Legacy Format Support

Best Practices

`any_of` (OR Logic)

`all_of` (AND Logic)