Org-Level EDH (AWS - CloudTrail Mode)

In addition to existing Event-Driven Harvesting (EDH) capabilities, InsightCloudSec includes support for Org-level (CloudTrail Mode) EDH. In the regular deployment, EDH augments standard harvesting methods, pulling data from AWS CloudWatch Events and AWS CloudTrail into a central Eventbus for InsightCloudSec's consumption. More information on standard EDH, including a detailed diagram, is included on the Event-Driven Harvesting (AWS) page. Org-Level EDH behaves in the same way as the existing EDH but with two key differences:

• Org-Level EDH typically delivers events within 5 minutes (on average)
  • This delivery time is dependent upon the AWS CloudTrail service-level agreement (SLA). Review the AWS CloudTrail documentation for more information.
• Org-Level EDH does not require additional manual configuration when new cloud accounts are added.
  • Any new cloud accounts added to your overall footprint are automatically discovered and included in your EDH setup. There are no complex configuration requirements to connect new cloud accounts and ensure the associated resources are harvested.

Deployment Diagram

Prerequisites

• A functioning InsightCloudSec Platform
• AWS Organization(s) added to InsightCloudSec
• InsightCloudSec Admin permissions
• Basic familiarity with Terraform and the general capabilities.
  • If you are not familiar with Terraform we are happy to assist with this configuration, reach out to us through the Customer Support Portal.
• Required Terraform template(s) (provided below)
  • For customers that do not anticipate using Terraform to deploy, reach out to us for assistance in creating the files required to establish this configuration
• Admin access to your AWS Master/Payer Account.
  • We strongly recommend installing Org-Level EDH inside of the master/payer AWS account and not in the same account that you are using to run your InsightCloudSec platform (this avoids conflicting issues around permissions during configuration)

Required Permissions

Required Templates

Configuring Org-Level EDH

Terraform Setup

Before continuing with deployment, ensure you have downloaded the required templates and have the appropriate permissions to execute the required Terraform commands. The following template (ics-edh.auto.tfvars) is provided as an example, so you will need to update it to reflect the configuration for your specific needs.

terraform
1
//
2
// COMMON
3
//
4
// Name prefix for newly created resources
5
name_prefix = "rapid7-ics-edh"
6
// Add custom tags to newly created resources
7
/*
8
custom_tags = {
9
    Product     = "Rapid7-ICS"
10
    Environment = "Production"
11
}
12
*/
13
// END COMMON
14
//
15
// CLOUDTRAIL SOURCE
16
//
17
// Use existing CloudTrail or create new CloudTrail via Terraform? (A new CloudTrail is *not* required for EDH and it is uncommon to create one)
18
use_existing_cloudtrail = true
19
// Account ID where existing CloudTrail is deployed (if `use_existing_cloudtrail = true`) or will be created (if `use_existing_cloudtrail = false`)
20
source_account = "111111111111"
21
// "Home region" where existing CloudTrail is deployed (if `use_existing_cloudtrail = true`) or will be created (if `use_existing_cloudtrail = false`)
22
source_region = "us-east-1"
23
// If not creating new CloudTrail, define account ID(s) where CloudTrail is configured (comment out if `use_existing_cloudtrail = false`)
24
// Some customers use multiple S3 buckets in multiple accounts for CloudTrail storage. If applicable, enter multiple account IDs where CloudTrail is configured (this is *not* common; most customers have a single org-level CloudTrail configured in a single account).
25
// Otherwise, enter a single account ID for the single bucket above.
26
cloudtrail_accounts = ["1111111111111"]
27
// Existing CloudTrail bucket name(s) (NOT ARN, JUST BUCKETNAME); comment out if `use_existing_cloudtrail = false`
28
// Some customers use multiple S3 buckets for CloudTrail storage. If applicable, enter multiple bucket names (this is *not* common; most customers have a single org-level CloudTrail S3 bucket in a single account).
29
existing_cloudtrail_buckets = ["aws-cloudtrail-logs-111111111111-065d972d"]
30
// Some customers have an existing CloudTrail with an existing "SNS notification delivery" target already in place.
31
// If this is the case, and you do NOT wish to replace the existing "SNS notification delivery" target, set this value to `true`.
32
// Setting this value to `true` will instead create an "Event notification" on each of the `existing_cloudtrail_buckets`.
33
// This "s3:ObjectCreated:*" event will target the SNS topic created in the `target_account`.
34
//
35
// If you *do* want to replace your existing "SNS notification delivery" topic, you will need to manually update the target/topic on the existing CloudTrail with the SNS ARN output from `sns_topic_arn` after the `terraform apply` operation
36
s3_sns_trigger = false
37
// END CLOUDTRAIL SOURCE
38
//
39
// CLOUDTRAIL TARGET
40
//
41
// Account ID where CloudTrail events will be forwarded and consumed via the SQS and Lambda resources deployed by this template. 
42
// This is typically the same account ID as the `source_account`, but the SQS and Lambda resources deployed can be deployed to another account.
43
target_account = "111111111111"
44
// Region where CloudTrail events will be forwarded and consumed via the SQS and Lambda resources deployed by this template.
45
target_region = "us-east-1"
46
// Existing ICS harvesting role name (not ARN) deployed in the `target_account`. Additional permissions will be attached to this role to allow access to SQS
47
existing_harvesting_role = "rapid7-consolidated"
48
// Customer-provided SNS topic ARN(s) to notify for Lambda and SQS errors
49
// This is not required, but RECOMMENDED in order to receive alerts for Lambda and/or SQS errors
50
//alarm_actions = ["arn:aws:sns:us-east-1:111111111111:sns-topic-name"]
51
// END CLOUDTRAIL TARGET

InsightCloudSec Setup

Refer to the steps below to configure Org-Level EDH in InsightCloudSec. In general, we do not recommend changing an existing setup or creating a new setup to use Org-Level EDH that would compete with an existing setup.

1. Navigate to Cloud > Cloud Accounts and select EDH Consumers.
2. Click EDH Configuration, then click AWS SQS Consumer.
3. Complete this form by selecting the Consumer Account (already onboarded in InsightCloudSec) from the drop-down list. You will also need to do the following:
   1. Provide the ARN for the SQS Consumer First-In-First-Out (FIFO) queue (not the capture queue).
   2. Select the CloudTrail via Lambda from the Producer Type drop-down menu.
   3. Click Configure. The Consumer will remain in a pending state while setup is in progress.