Splunk

Splunk is a log aggregator that allows you to pull in logs from across your network environment for querying and reporting. By connecting Splunk and InsightIDR, you can monitor the logs you have sent to Splunk in InsightIDR.

You can send data using either TCP or UDP, depending on how you want to configure the log forwarding.

Before You Begin

This procedure is intended for Splunk version 7.2.0.

When configuring Splunk to send data, you can send only a single stream of data per port on the Collector. Therefore, you need to configure a new port for each respective data type you send.

If you attempt to send all of your Splunk data to InsightIDR over one port, it will not parse out the data into different event sources. Each data type should have its own port.

If the correct ports are not open, your TCP processer can become overloaded and crash, which can crash your Splunk indexer and Exchange Data.

Edit Splunk Configuration Files

If you want to send firewall logs from one application and IDS logs from another application, a Splunk admin must write a stanza for each of these applications and specify that each log stream goes to an individual port. For organizations that want to send multiple application logs to InsightIDR, this method can become inefficient.

If you're a Splunk admin, you can configure Splunk to send data using TCP. Using syslog may cause Splunk to fail during data collection, but if you need to use syslog, follow these instructions: https://docs.splunk.com/Documentation/Splunk/7.2.4/Forwarding/Forwarddatatothird-partysystemsd#Syslog_data

For additional information on using the Splunk btool to complete this configuration, see this Splunk forum post: https://answers.splunk.com/answers/729325/has-anyone-been-able-to-integrate-splunk-with-insi.html?childToView=729951#answer-729951

When configuring the .conf files, you need to include the following code blocks:

Splunk standard practice indicates all configuration should be completed in app local directory. However, any Syslog configuration should be completed in System local directory.

app.conf

 
1
[install]
2
is_configured = 1

idrsetup.conf

 
1
[setupentity]
2
hostnames = <IP>

Do not include a port with the IP address.

indexes.conf

Delete the contents of this file upon installation.

outputs.conf

 
1
[tcpout:rapidreader]
2
server = IP:PORT
3
sendCookedData = false
4


5
[tcpout]
6
defaultGroup = rapidreader
7
indexAndForward = true
8
forwardedindex.0.denylist = ^((?!alerts|cyberark).)*$
9

In the last line of the code block, ^((?!alerts|cyberark).)*$ represents the regex used to deny anything except for logs containing "alerts" or "cyberark." You can use regex or build an expression to block certain words as necessary.

props.conf

 
1
[index::cyberark]
2
TRANSFORMS-cyberark = rapid
3
[host::MYHOST]
4
TRANSFORMS-myhost = rapid

The above code block contains two items forwarded to InsightIDR by both index and the host.

transforms.conf

 
1
[rapid]
2
REGEX = .
3
DEST_KEY = _TCP_ROUTING
4
FORMAT = rapidreader

If you are using syslog, use the following:

 
1
[rapid]
2
REGEX = .
3
DEST_KEY =  _SYSLOG_ROUTING
4
FORMAT = rapidreader

Configure InsightIDR to use the log aggregator collection method

After you configure the log aggregator to collect data, select Log Aggregator as your collection method. Ensure that you your chosen event source supports this collection method. To set up your event source, follow the relevant documentation.

To configure the log aggregator collection method in InsightIDR:

  1. From the left menu, go to Data Collection and click Setup Event Source > Add Event Source.
  2. Do one of the following:
    • Search for an event source in the event sources search bar.
    • In the Product Type filter, select your desired product type.
  3. Select an event source that supports this collection method.
  4. Name the event source. The name you enter will be used for the log that the event data streams into in Log Search. If you do not name the event source, the log name will be the event source's default name.
  5. Select a collector.
  6. Optionally, choose to send unparsed data.
  7. Select Log Aggregator as your collection method.
  8. Select Splunk from the log aggregator dropdown.
  9. Enter the port from Splunk that you configured to accept logs.
  10. Select TCP from the protocol dropdown. Optionally, choose to encrypt data.
  11. Click Save.