Org-Level EDH (AWS - CloudTrail Mode)

In addition to existing Event-Driven Harvesting (EDH) capabilities, InsightCloudSec includes support for Org-level (CloudTrail Mode) EDH. In the regular deployment, EDH augments standard harvesting methods, pulling data from AWS CloudWatch Events and AWS CloudTrail into a central Eventbus for InsightCloudSec's consumption. More information on standard EDH, including a detailed diagram, is included on the Event-Driven Harvesting (AWS) page. Org-Level EDH behaves in the same way as the existing EDH but with two key differences:

  • Org-Level EDH typically delivers events within 5 minutes (on average)
  • Org-Level EDH does not require additional manual configuration when new cloud accounts are added.
    • Any new cloud accounts added to your overall footprint are automatically discovered and included in your EDH setup. There are no complex configuration requirements to connect new cloud accounts and ensure the associated resources are harvested.

Getting Started with Org-Level EDH

Event-Driven Harvesting is an advanced and complex feature. Before getting started, we recommend reaching out to us through the Customer Support Portal to ensure that you have the best possible experience getting started with EDH.

Deployment Diagram

Org-Level EDH Deployment

Prerequisites

  • A functioning InsightCloudSec Platform
  • AWS Organization(s) added to InsightCloudSec
  • InsightCloudSec Admin permissions
  • Basic familiarity with Terraform and the general capabilities.
    • If you are not familiar with Terraform we are happy to assist with this configuration, reach out to us through the Customer Support Portal.
  • Required Terraform template(s) (provided below)
    • For customers that do not anticipate using Terraform to deploy, reach out to us for assistance in creating the files required to establish this configuration
  • Admin access to your AWS Master/Payer Account.
    • We strongly recommend installing Org-Level EDH inside of the master/payer AWS account and not in the same account that you are using to run your InsightCloudSec platform (this avoids conflicting issues around permissions during configuration)

Product name to be replaced

You may observe that some components, screen captures, or examples use our former product name, DivvyCloud. This doesn't affect the configuration or the product's functionality, and we will notify you as we replace these component names.

Required Permissions

Required Templates

Configuring Org-Level EDH

Terraform Setup (v0.12.x required)

For this part of the deployment we assume that you have downloaded the required templates and have the appropriate permissions to execute the required Terraform commands.

Variables.tf

The template below is provided as an example and will need to be updated to reflect the configuration for your specific needs.

terraform
1
// AWS master/payer account
2
variable "aws_account_id" {
3
type = string
4
default = "XXXXXXXXXXXX"
5
}
6
// AWS target region
7
variable "region" {
8
type = string
9
default = "us-east-1"
10
}
11
// Existing harvesting role in master/payer account
12
variable "existing_harvesting_role" {
13
type = string
14
default = "DivvyCloud-Standard-Role"
15
}
16
// If set to false, CloudTrail and associated S3 bucket are not created
17
variable "create_cloudtrail" {
18
type = bool
19
default = false
20
}
21
// If `create_cloudtrail` is false, specify existing CloudTrail S3 bucket and associated SNS topic name
22
variable "existing_cloudtrail_bucket" {
23
type = string
24
default = "arn:aws:s3:::EXISTING-CLOUDTRAIL-BUCKET-NAME"
25
}
26
// Update `arn:aws:sns:REGION:ACCOUNT-ID:DivvyCloud-EDH-Org-CloudTrail` in provided IAM policy to reference existing SNS topic
27
variable "existing_cloudtrail_topic" {
28
type = string
29
default = "EXISTING-SNS-TOPIC-NAME"
30
}

InsightCloudSec Setup

Refer to the steps below to configure Org-Level EDH in InsightCloudSec. In general, we do not recommend changing an existing setup or creating a new setup to use Org-Level EDH that would compete with an existing setup.

  1. Navigate to Cloud > Cloud Accounts and select EDH Consumers.
  2. Click EDH Configuration, then click AWS SQS Consumer.
  3. Complete this form by selecting the Consumer Account (already onboarded in InsightCloudSec) from the drop-down list. You will also need to do the following:
    1. Provide the ARN for the SQS Consumer First-In-First-Out (FIFO) queue (not the capture queue).
    2. Select the CloudTrail via Lambda from the Producer Type drop-down menu.
    3. Click Configure. The Consumer will remain in a pending state while setup is in progress.