Enrich Data with Open Source Plugins

The Enrich Alert Data with Open Source Plugins workflow allows InsightIDR to gather additional content to enrich and further contextualize your investigations and alerts. The workflow takes the evidence from your investigations and uses third-party, open source tools and plugins to acquire more extensive information that may be helpful.

Tip: Configure a trigger

This workflow is designed to accommodate almost any legacy detection rule (formerly known as User Behavior Analytics detection rules) in InsightIDR and is capable of enriching alert data as long as valid input is available. As a best practice, Rapid7 highly recommends that you configure this workflow to run automatically using a trigger, so that you can see enrichment results as soon as you open an investigation for the first time.

InsightIDR and InsightConnect - Better Together

The Enrich Alert Data with Open Source Plugins workflow is a prime example of how Security Information and Event Management (SIEM) and Security Orchestration And Response (SOAR) systems can work together to tackle security issues as they emerge in your environment. As you review this document for workflow details and configuration instructions, consider how SOAR can blend into your security processes and practices. Automating some of the steps that your security team currently has to perform manually greatly improves their efficiency when it comes to threat response.

Used together, InsightIDR and InsightConnect can help facilitate these efficient security processes and practices. To this end, Rapid offers a practical playbook guide that details examples of common workflow use cases.

Bear in mind that data enrichment is only the first step you can take towards energizing your incident response program. If you want to further automate the custom security actions that enriched data reveals as necessary, you can do so with InsightConnect.

How This Enrichment Workflow Works

The workflow ingests data from InsightIDR and sends this information to the Insight Orchestrator for InsightConnect to process. The workflow then runs the data through the following tasks in this order:

  1. IP address lookup
  2. Domain lookup
  3. URL check
  4. Malicious processes and hash check

Tip

The workflow will attempt to process any and all indicators in these categories that are available to it at the time of initiation. The workflow does not require input from all four categories to run.

The following free and open source plugins perform the data enrichment:

Plugin

Description

Unshorten URL

Attackers often obfuscate URLs with simple shortened variants to mislead investigators. The Unshorten URL plugin attempts to reverse this action. If the plugin is successful, or if the original URL was not shortened, the workflow will send the resulting domain substring to the Whois plugin described in this table.

ExtractIt

The ExtractIt plugin parses out the domain substring from the full URL to prepare it for domain analysis.

Team CYMRU - MHR

This plugin looks up files or hashes in the Malware Hash Registry by Team Cymru.

Dig

The Dig plugin performs forward and reverse DNS lookups.

Whois

As noted previously, this plugin uses the Linux WHOIS client to query domain names or IP addresses against registries stored in WHOIS servers.

Each loop of the workflow addresses different details and prints artifacts to display the outcomes of each loop.

IP Address Lookup Loop

The IP Address Loop uses the Whois and Dig InsightConnect plugins to gather data on IP addresses found by InsightIDR.

First, the loop runs an IP address through a whois lookup, then prints an artifact detailing the following for the IP address:

  • CIDR
  • Network name
  • Network type
  • Network range
  • Physical address
  • City
  • State
  • Postal code
  • Country
  • Registration date (returned registration dates that are less than seven days from the alert date are generally highly suspicious and warrant IP blocking consideration)
  • Update date
  • Organization
  • Organization name
  • Organization technician email address
  • Organization technician phone number
  • Organization abuse email address
  • Organization abuse phone number

Next, the loop runs the IP address through a reverse DNS lookup with Dig, which then prints an artifact with the Dig query details.

Domain Loop

The Domain Loop uses the Whois plugin to gather data on domains found by InsightIDR. The workflow then prints an artifact containing the following details for each domain:

  • Name
  • DNSSEC
  • Domain registrar
  • Nameservers
  • Update date
  • Creation date (similar to registration dates returned by the IP address lookup loop, returned creation dates that are less than seven days from the alert date are generally highly suspicious and warrant IP blocking consideration)
  • Domain status
  • Registrant country code
  • Registrar URL
  • Expiration date
  • Registrant name
  • Registrar IANA ID
  • Registry domain ID
  • Registrar WHOIS server
  • Registrar abuse contact email
  • Registrar abuse contact phone number

URL Loop

The URL Loop checks URLs found by InsightIDR, then loops over the URLs to unshorten the URL to its full form. The loop will inspect the unshortened URL for the domain of the URL to offer enriched analysis.

After the URL was evaluated, the URL domain is extracted and the Whois plugin is applied to gather domain data. The workflow then prints an artifact containing the following details for each domain:

  • Name
  • DNSSEC
  • Domain registrar
  • Nameservers
  • Update date
  • Creation date (similar to registration dates returned by the IP address lookup loop, returned creation dates that are less than seven days from the alert date are generally highly suspicious and warrant IP blocking consideration)
  • Domain status
  • Registrant country code
  • Registrar URL
  • Expiration date
  • Registrant name
  • Registrar IANA ID
  • Registry domain ID
  • Registrar WHOIS server
  • Registrar abuse contact email
  • Registrar abuse contact phone number

Processes Loop

The Processes Loop ingests a list of processes found by InsightIDR, then loops over the hash content for each process discovered. The nested Hashes Loop runs a lookup with the Team Cymru MHR plugin to gather the following details for each hash of a process:

  • Hash
  • Hash algorithm/type
  • Lookup result
  • Anti-virus detection percentage
  • Last seen timestamp
  • Response code

NOTE

As a general guideline, a returned response code with a value of 200 (meaning the process was found successfully) and a returned last seen timestamp that is less than seven days old indicates a highly suspicious process.

How to Configure This Workflow

You can create triggers to automatically run this enrichment workflow for certain detection rules, or manually run the enrichment workflow directly from an investigation.

To configure a trigger for this enrichment workflow:

  • See the triggers page for overview information and configuration instructions.

To manually run this enrichment workflow:

  1. From your InsightIDR homepage, select Investigations from the left menu.
  2. Open the desired investigation. You will see a timeline of events involving the user.
  3. Click the Take Action button. The “Take Action” panel appears.
  4. From the “Select an Action Category” dropdown, click Enrichment Workflows.
  5. From the “Select an Automation Action to Take” dropdown, click Enrich Alert Data with Open Source Plugins. Click Continue.
    • Workflows that accept multiple objects appear with gray-colored tags. Hover over this tag to see what objects this workflow accepts as input. Since this workflow accepts multiple input types, it will display this gray tag.
    • If you are configuring a new workflow, make sure you select the template version of Enrich Alert Data with Open Source Plugins. Templates are indicated by a small Rapid7 logo next to the workflow name.
  6. Give your workflow a name.
  7. Select connections for each of the workflow’s plugins. Click Continue when ready.
  8. Configure the details that InsightIDR will pass to the workflow for enrichment. You can select from applicable data points from each displayed dropdown menu.
  9. Click Take Action when finished.

The workflow events will appear on the Investigation timeline as the processes complete.