InsightOps - Configure the Insight Agent to Send Logs

The Insight Agent sends asset log data to the Insight platform using a special configuration file called logging.json. By default, the logging.json file instructs the agent to send logs directly to the data.logs.insight.rapid7.com endpoint according to your region, but you can configure a proxy destination if necessary.

Prerequisites

To use log forwarding through the logging.json file, you must have access to an API key that allows you to create logs and logsets.

Ensure you have the correct permissions:

1. Create a new user account.
2. Assign the user to only InsightOps.
3. Restrict access to Log Search operations by selecting the Log Search Insight Agent Service Role permission.
4. Generate an API key.

Proxy Configuration

If you prefer to send asset log data through a proxy instead of directly to the Insight platform, you can do so by adding the following additional key-value pairs in the overall config object of your logging.json file:

• proxy-type - Specifies the application protocol. At this time, only HTTP is supported.
• proxy-url - Specifies the address of your proxy server. This can be a hostname or IP address.
• proxy-port - Specifies the port that the Insight Agent will use to communicate with your proxy server.

This example shows how you should format your proxy definition in logging.json:

{
  "config": {
    "proxy-type": "HTTP",
    "proxy-url": "10.7.1.219",
    "proxy-port": "3128",
    ...followed by the rest of your objects
   }
}

logging.json Creation Wizard

Use the Insight Agent data source wizard in InsightOps to automatically create and format a logging.json file that includes your selected API key and data region. You can download the complete file directly from the wizard afterwards.

To create your logging.json file with the wizard:

1. In InsightOps, click the Add Data button.
   • Alternatively, you can click the Add Data link located on either the Log Search or Data Collection tabs.
2. In the “Install the Insight Agent” section, click the operating system of the asset that you intend to install the agent on.

3. On the “Send Logs via Insight Agent” panel, choose to use your selected API key.
4. Paste your selected API key in the field and click Apply.

5. Verify that your formatted logging.json file includes your selected API key, your data region endpoint, and your region code. Note that the hostnames in the ‘destination’ fields can be changed as per requiremnents. Click Download to retrieve your file.
6. Place your logging.json file in one of the following directories on your asset according to its operating system:
   • Windows - C:\Program Files\Rapid7\Insight Agent\components\insight_agent\common\config
   • Mac and Linux - /opt/rapid7/ir_agent/components/insight_agent/common/config

7. Finally, restart the Insight Agent service.
   • On Windows, this service is called “Rapid7 Insight Agent” in your service manager.
   • On Mac and Linux, you can restart the service by running the sudo service ir_agent restart command in a terminal.

logging.json Formatting Templates

If you prefer to create your logging.json file manually, see the following templates.

Mac and Linux

The Mac and Linux version of logging.json has the following structure:

{
  "config": {
		"name": "A label for your log following configuration",
		"endpoint": "eu.data.logs.insight.rapid7.com",
		"region": "eu",
		"api-key": "Your API key",
		"state-file": "/opt/rapid7/ir_agent/components/insight_agent/common/config/logs.state",
		"formatter" : "plain",
		"metrics": {
			"system-stat-token": "your log token here",
			"system-stat-enabled": true,
			"metrics-interval": "60s",
			"metrics-cpu": "system",
			"metrics-vcpu": "core",
			"metrics-mem": "system",
			"metrics-swap": "system",
			"metrics-net": "sum eth0",
			"metrics-disk": "sum sda4 sda5",
			"metrics-space": "/"
		},
    "logs": [
        {
          "name": "System log",
          "destination": "your-server-name/syslog",
          "path": "/var/log/syslog",
          "enabled": true
        },
        {
          "name": "Kernel",
          "destination": "your-server-name/kernel",
          "path": "/var/log/kern.log",
          "enabled": true
        },
        {
          "name": "Authentication",
          "destination": "your-server-name/auth",
          "path": "/var/log/auth.log",
          "enabled": true
        },
        {
          "name": "Insight Agent log",
          "token": "your log token here",
          "path": "/opt/rapid7/ir_agent/components/insight_agent/common/agent.log",
          "enabled": false
        }]
  }
}

Windows

The Windows version of logging.json has the following structure:

{
  "config": {
    "name": "A label for your log following configuration",
    "endpoint": "eu.data.logs.insight.rapid7.com",
    "region": "eu",
    "api-key": "Your API key",
    "state-file": "C:\\Program Files\\Rapid7\\Insight Agent\\components\\insight_agent\\common\\state.file",
    "formatter" : "plain",
    "metrics": {
      "system-stat-token": "your log token here",
      "system-stat-enabled": true,
      "metrics-interval": "60s",
      "metrics-cpu": "system",
      "metrics-vcpu": "core",
      "metrics-mem": "system",
      "metrics-swap": "system",
      "metrics-net": "sum eth0",
      "metrics-disk": "sum sda4 sda5",
      "metrics-space": "/"
    },
    "windows-eventlog": {
      "enabled": true,
      "token": "cfe50cc8-7651-42f3-ac23-866bf87a6aad"
      },
     "logs": [
     {
       "name": "Custom log",
       "destination": "your-server-name/custom log",
       "path": "c:\\logs\\mylogfile*.log",
       "enabled": true
     },
     {
       "name": "Insight Agent log",
       "token": "your log token here",
       "path": "C:\\Program Files\\Rapid7\\Insight Agent\\components\\insight_agent\\common\\agent.log",
       "enabled": false
     }]
  }
}

Object legend