InsightVM Technology Add-On for Splunk

The InsightVM Technology Add-On for Splunk allows you to import InsightVM asset and vulnerability data into your Splunk environment and view that data with the companion InsightVM Dashboard. For InsightVM customers, these new Splunk import and visualization tools functionally replace the older Nexpose Technology Add-On and Nexpose Dashboard.

The new InsightVM edition of this add-on differs from its predecessor in the following ways:

Installation

You can install the add-on as an app using the Splunk UI:

1. From the Apps menu in Splunk, select Manage Apps.
2. Select Browse More Apps.
3. Search for the "Rapid7 InsightVM Technology Add-On".
4. Select Install from the app listing.
5. When prompted, restart Splunk to finish.

Configuration

You must configure two components in the add-on before you can use it:

• The connection
• One or more inputs

Connection

The connection requires an API key for connecting to and retrieving data from InsightVM. The add-on allows you to create multiple connections if you want to use different connections across different inputs.

If you need to generate a new API key for a connection, follow these steps:

1. Sign in to the Insight Platform.
2. Select the gear icon in the top menu and click API Keys.
3. Select Organization Key.
4. Select + New Key.
5. Enter a name for the key and click Generate.
6. Copy and store the generated key in a secure location.

After you have an API key, follow these steps to configure the InsightVM connection in Splunk.

1. Navigate to the Rapid7 InsightVM Technology Add-On available under the Apps menu in Splunk.
2. Select Configuration.
3. Select Add.
4. Enter a name for the connection.
5. Enter your region, which is a two-character string based on your location (such as us).
   • Additional region information is available in the Supported Regions section of the Product APIs page.
6. Enter your generated API key.
7. Click Add.

Inputs

The add-on uses inputs to configure the import of asset and vulnerability data from InsightVM. These inputs allow you to further filter the data that Splunk retrieves and ingests.

There are two inputs available for configuration:

• InsightVM Asset Import - Imports asset and optional vulnerability finding data.
• InsightVM Vulnerability Definition Import - Imports vulnerability definitions.

To create a new input, navigate to the add-on menu and click Inputs > Create New Input and select the one you want to configure.

InsightVM Asset Import

Customizable fields for the InsightVM Asset Import input are as follows:

Example asset filters

The following queries are examples of what you can set for the Asset Filter field:

• sites IN [`site-name]
• tags IN [tag-name]
• os_family = Windows

Example vulnerability filters

In similar fashion, these queries are examples for the Vulnerability Filter field:

• cvss_v2_score > 6
• severity = Critical

InsightVM Vulnerability Definition Import

The add-on uses this input to import vulnerability definitions from InsightVM. You can use this data to correlate with vulnerability findings (if you elect to import those) as well. Importing vulnerability definitions is not required for visualizing asset findings in your environment, but it does provide additional details about the vulnerabilities.

Customizable fields for the InsightVM Vulnerability Definition Import input are as follows:

Imported Data

The following tables indicate imported data fields and contain example data for each.

Asset Sourcetype

Asset Vulnerability Finding Sourcetype

Vulnerability Definition Sourcetype

The Vulnerability Definition sourcetype is not currently used by the InsightVM Dashboard, but the imported data is made available to supplement the asset vulnerability findings imported with the Asset Import input.

Visualizing Data

The InsightVM Dashboard for Splunk serves as a starting point for visualizing data imported with the add-on. There are two dashboards included:

• The InsightVM Assets dashboard used for visualizing asset details
• The InsightVM Vulnerability Findings dashboard used for visualizing details of vulnerability instances found within assets.

The Rapid7 InsightVM Dashboard features several customization options and you can install it as an app in the same way as the add-on.

FAQs

Is there a migration path from the Nexpose Technology Add-On to this new Add-On for InsightVM?

No. Due to the differences in how each add-on retrieves data and the content that is stored, there is not a migration path from the legacy Nexpose Technology Add-On to the InsightVM Technology Add-On. However, both add-ons can be used at once so it is possible to continue using the Nexpose Technology Add-On until you are ready to fully disable it.

Why am I not seeing any data in my add-on/dashboard?

Check the selected index and time period for filtering data. These often need to be adjusted to filter correctly for assets and vulnerabilities. In addition, if the default rapid7 index was used for the inputs, make sure this index has already been created.

Does the Asset Import input import all assets each time it runs?

No. When the asset import is run for the very first time it will import all assets. After that, all subsequent imports will only pull in assets that have been newly scanned since the last import occurred.

Does the Vulnerability Definition Import input import all definitions each time it runs?

Yes, all vulnerability definitions will be imported each time it is run. For this reason, we recommend running this import at a maximum of once per day.

How do I know if a vulnerability is new as opposed to remediated?

Check the finding_status field of a vulnerability finding to determine whether it's new, remediated, or unchanged. The new and remediated statuses indicate new and remediated vulnerabilities respectively, while the found status indicates a previously found, unchanged vulnerability finding.

Can I identify whether a vulnerability has been remediated?

Yes, all vulnerability findings will have a finding_status when the add-on imports them into Splunk. If a vulnerability is remediated, the add-on imports it as a new event with a finding_status of remediated.

Can the add-on import all data every time an input runs?

No, the add-on currently only imports data for assets that have been scanned since the last time the input ran. Internally, Rapid7 tracks the last time a job was successful and it only updates when no errors are received to avoid losing data that should be imported.

In the logs I can see one or more requests failed (such as a status 504), does this mean the import process will miss data?

Not all HTTP requests succeed. For this reason, when a request fails the application will retry making the request up to a maximum of 3 times. A delay is added in between each retry, and will increase in duration for subsequent retries. If the maximum number of retries has been reached and the request has not succeeded, the import will fail and start over next time the Data Input is executed.

If the issue is persistent, confirm that the cause is not due to a proxy or firewall rules. Otherwise, contact Rapid7 Support to ensure there is no service degradation.

Why are some of my vulnerability definition property values truncated?

Only the largest vulnerability definitions (not assets or vulnerability findings) are affected by this. Some vulnerability definitions are so large they may cause performance issues and require that some data is removed. Truncated strings will end with an ellipsis (...) and truncated lists will be capped at a length of 75.

I see truncation warnings in the splunkd.log, what does this mean?

The warnings may look similar to LineBreakingProcessor - Truncating line because limit of XXXXX bytes has been exceeded.

Splunk truncates events that are too large. This can result in data being excluded from the index. If these errors are present in the logs for rapid7:insightvm:asset or rapid7:insightvm:asset:vulnerability_finding data sourcetypes, then the TRUNCATE value for the relevant sourcetype should be increased in the Technology Add-On's app.conf and a new initial import performed.

If I previously chose not to import vulnerability findings but I want to now, will my future runs import the missed vulnerabilities?

No, a new initial import will need to be performed to pick up vulnerability findings that were previously not imported.

If I suspect data has been missed during an import run, what can I do?

Ensure that you're using the latest version of the add-on. Any data loss issues you are experiencing may be resolved by defect fixes that have been applied in subsequent versions. When upgrading to a newer version of the add-on, it may be necessary to clear the state data stored by the previous version since this could affect future imports. See the Upgrading section for more information.

Upgrading

You can upgrade the add-on by clicking the Update button from the Splunk App Browser page or Update Available from the App Management page. For additional information on the App update process for Splunk Cloud, refer to the Splunk documentation at https://docs.splunk.com/Documentation/AddOns/released/Overview/SplunkCloudinstall.

It's not always necessary to perform a full import when upgrading, but if the upgrade is because data has been missed during a previous import then a full import should be performed using the latest version of the add-on. The add-on decides whether a full import or partial import ( a "partial import" refers to differences since last import) should be performed using state information it stores in Splunk's KV store.

It's important to note that performing a full import may result in a lot of duplicate data being indexed if a new index is not created or the existing index is not refreshed.

There are a number of ways to force a full import. Rapid7 recommends that you read through all the steps before starting the process.

Method 1: Creating a new add-on input

This will not remove any existing data or add-on configuration. Instead, a second add-on input is created. The state data stored in Splunk's KV store is tied to the add-on input name, so creating an input with a different name means the add-on performs a full import when the new input is executed by Splunk.

1. (Optional) Create a new index by going to Settings > Indexes > New. Enter a unique name that differs from the previous index name (such as rapid7_new).
2. From the Splunk home page, select Rapid7 InsightVM from the Apps menu.
3. Select Create New Input and enter a unique name that is different from the name chosen during the original input creation.
4. Make sure to select the appropriate index.
5. An initial import will be performed the next time the import executes. This may result in a lot of duplicate data being indexed if a new index was not created. Consider disabling the original input to mitigate this condition.

Method 2: Manually deleting the state data

This will remove the existing state data which may not be recovered. As previously mentioned, the name of the state data is tied to the name of the input.

1. Find the name of the input by clicking on the Rapid7 InsightVM Splunk App and noting the name of the input, for example "asset_import". This is the {{input_name}} used during step 3 below.
2. (Optional) View the values of all state data by sending a GET request to {{splunkdProtocol}}://{{splunkHost}}:{{splunkdPort}}/servicesNS/nobody/TA-rapid7-insightvm/storage/collections/data/TA_rapid7_insightvm_checkpointer. Don't forget to replace the placeholders with values appropriate for your environment, for example https://localhost:8090/servicesNS/nobody/TA-rapid7-insightvm/storage/collections/data/TA_rapid7_insightvm_checkpointer. Headers that should be included on the request are Content-Type: application/json and Authorization: {{Basic Auth}}. The basic authorization uses the Splunk username/password.
3. Send DELETE requests to remove data for the chosen input. Don't forget to replace the placeholders with values appropriate for your environment. Headers that should be included on the request are Content-Type: application/json and Authorization: {{Basic Auth}}. The basic authorization uses the Splunk username/password.
   • {{splunkdProtocol}}://{{splunkHost}}:{{splunkdPort}}/servicesNS/nobody/TA-rapid7-insightvm/storage/collections/data/TA_rapid7_insightvm_checkpointer/{{input_name}}-last_import_time
   • {{splunkdProtocol}}://{{splunkHost}}:{{splunkdPort}}/servicesNS/nobody/TA-rapid7-insightvm/storage/collections/data/TA_rapid7_insightvm_checkpointer/{{input_name}}-last_assessed_time
4. An initial import will be performed the next time the import executes.

Troubleshooting

Three log files are available to help debug issues, usually located at <splunk_home>/var/log/splunk/:

• splunkd.log - Splunk general log
• ta_rapid7_insightvm_insightvm_asset_import.log - Log for the InsightVM Asset Import input
• ta_rapid7_insightvm_insightvm_vulnerability_definition_import.log - Log for the InsightVM Vulnerability Definition Import input