Cloudflare

InsightIDR can now parse logs from Cloudflare. The Cloudflare event source has been set up to use an Amazon S3 bucket. Using your own Amazon S3 bucket, you can now send logs to this bucket using Cloudflare's Logpush mechanism. Our collectors will then consume the logs from this bucket before pushing them to IDR to be parsed.

Initially, InsightIDR will ingest Cloudflare logs and add them to the "Raw Logs" Logset. View an example of Cloudflare logs, in the log example section of this topic.

To set up Cloudflare, you’ll need to:

  1. Review the Requirements
  2. Set up Cloudflare in InsightIDR
  3. Verify the Configuration

We also provide Log Examples.

Review the Requirements

In order to configure Cloudflare to send data to InsightIDR, you must configure the AWS S3 Bucket to send messages to InsightIDR. You will need access to the Logpush API which is currently only available to the Enterprise tier of Cloudflare. From this point you will need to configure an Amazon S3 destination either through the Cloudflare dashboard or API. Once you see log files appearing in the S3 bucket, you’ll be able to set up the event source in InsightIDR.

Configure InsightIDR to collect data from the event source

After you complete the prerequisite steps and configure the event source to send data, you must add the event source in InsightIDR.

To configure the new event source in InsightIDR:

  1. From the left menu, go to Data Collection and click Setup Event Source > Add Event Source.
  2. Do one of the following:
    • Search for Cloudflare in the event sources search bar.
    • In the Product Type filter, select Cloud Service.
  3. Select the Cloudflare event source tile.
  4. Choose your collector and select Cloudflare as your event source. You also have the option to name your event source.
  5. If you are sending additional events beyond alerts, select the unfiltered logs checkbox.
  6. Specify the authentication and path information for the S3 bucket that was created by Cloudflare.
  7. Click Save.

Verify the Configuration

Complete these steps to view your logs and ensure events are being sent to the Collector.

  1. On the new event source that was just created, click View Raw Log. If you see log messages in the box, this shows that logs are flowing to the Collector.
  2. Click Log Search in the left menu.
  3. Select the applicable Log Sets and the Log Names within them. The Log Name will be the event source name or Cloudflare if you did not name the event source. Cloudflare logs flow into the Raw Logs log set.

Logs take a minimum of 7 minutes to appear in Log Search

Please note that logs take at least 7 minutes to appear in Log Search after you set up the event source. If you see log messages when you select View Raw Log on the event source but do not see any log messages in Log Search after waiting for a few minutes for them to appear, then your logs do not match the recommended format and type for this event source.

Log Example

 
1
{
2
  "Datetime": "2022-08-15T01:40:59Z",
3
  "Action": "allow",
4
  "Kind": "firewall",
5
  "Source": "firewallrules",
6
  "ClientIP": "123.123.123.123",
7
  "ClientASNDescription": "R7-RAPID7",
8
  "ClientIPClass": "noRecord",
9
  "ClientCountry": "us",
10
  "ClientASN": 14618,
11
  "ClientRequestMethod": "GET",
12
  "ClientRequestUserAgent": "Rapid7 - Mobile",
13
  "ClientRequestPath": "/client/request/path/",
14
  "ClientRequestQuery": "?key=value",
15
  "ClientRequestScheme": "https",
16
  "ClientRequestHost": "www.rapid7.com",
17
  "ClientRefererHost": "www.rapid7.com",
18
  "ClientRefererPath": "/client/reference/path/",
19
  "ClientRefererQuery": "",
20
  "ClientRefererScheme": "https",
21
  "EdgeColoCode": "IAD",
22
  "EdgeResponseStatus": 200,
23
  "RuleID": "e8d36d34b4ed4b718acfdfeb30f64b2b",
24
  "MessageRule": "",
25
  "OriginatorRayID": "00",
26
  "RayID": "73ae39ad9c3182e6",
27
  "MatchIndex": 0
28
}