Enrich Data with Open Source Plugins
The Enrich Alert Data with Open Source Plugins workflow allows InsightIDR to gather additional content to enrich and further contextualize your investigations and alerts. The workflow takes the evidence from your investigations and uses third-party, open source tools and plugins to acquire more extensive information that may be helpful.
Tip: Configure a trigger
This workflow is designed to accommodate almost any legacy detection rule (formerly known as User Behavior Analytics detection rules) in InsightIDR and is capable of enriching alert data as long as valid input is available. As a best practice, Rapid7 highly recommends that you configure this workflow to run automatically using a trigger, so that you can see enrichment results as soon as you open an investigation for the first time.
InsightIDR and InsightConnect - Better Together
The Enrich Alert Data with Open Source Plugins workflow is a prime example of how Security Information and Event Management (SIEM) and Security Orchestration And Response (SOAR) systems can work together to tackle security issues as they emerge in your environment. As you review this document for workflow details and configuration instructions, consider how SOAR can blend into your security processes and practices. Automating some of the steps that your security team currently has to perform manually greatly improves their efficiency when it comes to threat response.
Used together, InsightIDR and InsightConnect can help facilitate these efficient security processes and practices. To this end, Rapid offers a practical playbook guide that details examples of common workflow use cases.
Bear in mind that data enrichment is only the first step you can take towards energizing your incident response program. If you want to further automate the custom security actions that enriched data reveals as necessary, you can do so with InsightConnect.
How This Enrichment Workflow Works
The workflow ingests data from InsightIDR and sends this information to the Insight Orchestrator for InsightConnect to process. The workflow then runs the data through the following tasks in this order:
Tip
The workflow will attempt to process any and all indicators in these categories that are available to it at the time of initiation. The workflow does not require input from all four categories to run.
The following free and open source plugins perform the data enrichment:
Plugin | Description |
|---|---|
Unshorten URL | Attackers often obfuscate URLs with simple shortened variants to mislead investigators. The Unshorten URL plugin attempts to reverse this action. If the plugin is successful, or if the original URL was not shortened, the workflow will send the resulting domain substring to the Whois plugin described in this table. |
ExtractIt | The ExtractIt plugin parses out the domain substring from the full URL to prepare it for domain analysis. |
Team CYMRU - MHR | This plugin looks up files or hashes in the Malware Hash Registry by Team Cymru. |
Dig | The Dig plugin performs forward and reverse DNS lookups. |
Whois | As noted previously, this plugin uses the Linux WHOIS client to query domain names or IP addresses against registries stored in WHOIS servers. |
Each loop of the workflow addresses different details and prints artifacts to display the outcomes of each loop.
IP Address Lookup Loop
The IP Address Loop uses the Whois and Dig InsightConnect plugins to gather data on IP addresses found by InsightIDR.
First, the loop runs an IP address through a whois lookup, then prints an artifact detailing the following for the IP address:
- CIDR
- Network name
- Network type
- Network range
- Physical address
- City
- State
- Postal code
- Country
- Registration date (returned registration dates that are less than seven days from the alert date are generally highly suspicious and warrant IP blocking consideration)
- Update date
- Organization
- Organization name
- Organization technician email address
- Organization technician phone number
- Organization abuse email address
- Organization abuse phone number
Next, the loop runs the IP address through a reverse DNS lookup with Dig, which then prints an artifact with the Dig query details.
Domain Loop
The Domain Loop uses the Whois plugin to gather data on domains found by InsightIDR. The workflow then prints an artifact containing the following details for each domain:
- Name
- DNSSEC
- Domain registrar
- Nameservers
- Update date
- Creation date (similar to registration dates returned by the IP address lookup loop, returned creation dates that are less than seven days from the alert date are generally highly suspicious and warrant IP blocking consideration)
- Domain status
- Registrant country code
- Registrar URL
- Expiration date
- Registrant name
- Registrar IANA ID
- Registry domain ID
- Registrar WHOIS server
- Registrar abuse contact email
- Registrar abuse contact phone number
URL Loop
The URL Loop checks URLs found by InsightIDR, then loops over the URLs to unshorten the URL to its full form. The loop will inspect the unshortened URL for the domain of the URL to offer enriched analysis.
After the URL was evaluated, the URL domain is extracted and the Whois plugin is applied to gather domain data. The workflow then prints an artifact containing the following details for each domain:
- Name
- DNSSEC
- Domain registrar
- Nameservers
- Update date
- Creation date (similar to registration dates returned by the IP address lookup loop, returned creation dates that are less than seven days from the alert date are generally highly suspicious and warrant IP blocking consideration)
- Domain status
- Registrant country code
- Registrar URL
- Expiration date
- Registrant name
- Registrar IANA ID
- Registry domain ID
- Registrar WHOIS server
- Registrar abuse contact email
- Registrar abuse contact phone number
Processes Loop
The Processes Loop ingests a list of processes found by InsightIDR, then loops over the hash content for each process discovered. The nested Hashes Loop runs a lookup with the Team Cymru MHR plugin to gather the following details for each hash of a process:
- Hash
- Hash algorithm/type
- Lookup result
- Anti-virus detection percentage
- Last seen timestamp
- Response code
NOTE
As a general guideline, a returned response code with a value of 200 (meaning the process was found successfully) and a returned last seen timestamp that is less than seven days old indicates a highly suspicious process.
How to Configure This Workflow
You can create triggers to automatically run this enrichment workflow for certain detection rules, or manually run the enrichment workflow directly from an investigation.
To configure a trigger for this enrichment workflow:
- See the triggers page for overview information and configuration instructions.
To manually run this enrichment workflow:
- From your InsightIDR homepage, select Investigations from the left menu.
- Open the desired investigation. You will see a timeline of events involving the user.
- Click the Take Action button. The “Take Action” panel appears.
- From the “Select an Action Category” dropdown, click Enrichment Workflows.
- From the “Select an Automation Action to Take” dropdown, click Enrich Alert Data with Open Source Plugins. Click Continue.
- Workflows that accept multiple objects appear with gray-colored tags. Hover over this tag to see what objects this workflow accepts as input. Since this workflow accepts multiple input types, it will display this gray tag.
- If you are configuring a new workflow, make sure you select the template version of Enrich Alert Data with Open Source Plugins. Templates are indicated by a small Rapid7 logo next to the workflow name.
- Give your workflow a name.
- Select connections for each of the workflow’s plugins. Click Continue when ready.
- Configure the details that InsightIDR will pass to the workflow for enrichment. You can select from applicable data points from each displayed dropdown menu.
- Click Take Action when finished.
The workflow events will appear on the Investigation timeline as the processes complete.