Configuring Cloud Anomaly Detection

Feature Not Yet Released

This page is intended for customers with early access to the Cloud Anomaly Detection feature, which is not generally available (GA) yet. We request that you do not share this URL/page outside of the Rapid7 organization or with customers that do not have early access.

This documentation resource is a work-in-progress; if you have questions, issues, or suggestions about the content provided here, we are happy to receive feedback.

For questions or issues, reach out to your CSM or to support through the Customer Support Portal.

Cloud Anomaly Detection can monitor Organizations/Accounts and Kubernetes Clusters but each are configured separately. Currently, Cloud Anomaly Detection only supports AWS.

  • For AWS Organizations and Accounts, Cloud Anomaly Detection relies on the AWS CloudTrail logs data that results from the standard InsightCloudSec harvesting process.
  • For EKS clusters, the EKS logs are streamed from CloudWatch to an S3 bucket using a Kinesis Firehose. InsightCloudSec then harvests the data from the S3 bucket.

Prerequisites

Before getting started you will need to ensure you have the following:

Organization and Account Configuration

AWS CloudTrail

AWS Cloud Anomaly Detection for organization and account monitoring uses CloudTrail logs inside S3 buckets. There are two main configuration scenarios:

  1. The primary harvesting role (from onboarding) is in the same account as the relevant CloudTrail bucket and/or the CloudTrail bucket is encrypted (in which case the primary harvesting role must be in the same account)
  2. The primary harvesting role (from onboarding) is in a different account than the relevant CloudTrail bucket

For convenience, InsightCloudSec has created an example S3 policy to use with the CloudTrail bucket:

Example S3 Policy

This policy grants the necessary S3 permissions to the new role for only the specified CloudTrail bucket and its contents.

json
1
{
2
  "Version": "2012-10-17",
3
  "Statement": [
4
    {
5
      "Sid": "AllowRapid7CloudTrailAccess",
6
      "Effect": "Allow",
7
      "Principal": {
8
        "AWS": "arn:aws:iam::<cloudtrail-account-id>:role/rapid7-cloud-anomaly-detection-role"
9
      },
10
      "Action": [
11
        "s3:GetObject",
12
        "s3:ListBucket"
13
      ],
14
      "Resource": [
15
        "arn:aws:s3:::<target-cloudtrail-bucket-name>",
16
        "arn:aws:s3:::<target-cloudtrail-bucket-name>/*"
17
      ]
18
    }
19
  ]
20
}

After performing AWS configuration, you'll need to configure the roles inside of InsightCloudSec.

S3 Deny Policies

You must also verify that no policies associated with the CloudTrail harvesting role contain deny statements for the required S3 permissions, otherwise Cloud Anomaly Detection will not work.

Same Account/Encrypted

If you followed the standard AWS onboarding process, the only required configuration item is ensuring the primary harvesting role can access the CloudTrail buckets. This is done by attaching a policy directly to the bucket.

  1. As an Admin, login to the AWS account containing the relevant primary harvesting role/CloudTrail bucket(s).
  2. Create a CloudTrail bucket policy for the primary harvesting role.
    1. Navigate to the S3 Dashboard.
    2. Click Buckets in the side navigation menu.
    3. Navigate to the CloudTrail bucket that needs to be monitored.
    4. Click Permissions.
    5. In the Bucket policy section, click Edit.
    6. Copy and paste the example S3 policy as a guide, ensuring you replace the placeholder values (the default name for the primary harvesting role is rapid7/rapid7-consolidated).
    7. Click Save Changes.
Different Accounts

With the primary harvesting role in place in a different account from the relevant CloudTrail buckets, you'll need to setup a role/trust relationship as well as an S3 bucket policy.

Example Trust Policy

This policy establishes a trust relationship between the primary harvesting role and the new CloudTrail bucket harvesting role.

External ID Recommended!

The external ID is optional but recommended. You can acquire a unique external ID for your InsightCloudSec instance from the Cloud Onboarding interface. See Onboard an AWS Account for more information.

json
1
{
2
  "Version": "2012-10-17",
3
  "Statement": [
4
    {
5
      "Sid": "TrustRapid7DeploymentAccount",
6
      "Effect": "Allow",
7
      "Action": "sts:AssumeRole",
8
      "Principal": "arn:aws:iam::<primary-harvesting-account>:role/<primary-harvesting-role-arn>",
9
      "Condition": {
10
        "StringEquals": {
11
          "sts:ExternalId": "<insightcloudsec-external-id>"
12
        }
13
      }
14
    }
15
  ]
16
}
  1. As an Admin, login to the AWS account containing the relevant CloudTrail bucket(s).
  2. Create a CloudTrail bucket policy for the new role.
    1. Navigate to the S3 Dashboard.
    2. Click Buckets in the side navigation menu.
    3. Navigate to the CloudTrail bucket that needs to be monitored.
    4. Click Permissions.
    5. In the Bucket policy section, click Edit.
    6. Copy and paste the example S3 policy as a guide, ensuring you replace the placeholder values.
    7. Click Save Changes.
  3. Create a role.
    1. Click Roles in the side navigation menu.
    2. Click Create role.
    3. Click AWS account, then click Another AWS account. Provide the Account ID for the account containing the primary harvesting role, then click Next.
    4. Search for and attach the CloudTrail bucket harvesting policy you just created, then click Next.
    5. Provide a name and description for the role. We recommend rapid7-cloud-anomaly-detection-role.
    6. Click Create role.
  4. Establish a trust relationship.
    1. Open the role you just created.
    2. Click Trust relationships.
    3. Click Edit trust policy.
    4. Copy and paste the example trust policy as a guide, ensuring you replace the placeholder values.

After the S3 bucket(s) has been successfully configured for Cloud Anomaly Detection, regular Harvesting cadence will pick up the logs and they will be displayed on the Cloud Anomaly Detection (Audit Log Monitoring) tab within the Cloud Accounts page. By default, the CloudTrail bucket and primary Harvester roles (from Onboarding) are used in Cloud Anomaly Detection. In cases where a different bucket or role are used, you can modify the default storage configuration. See Storage Configuration for details.

Kubernetes Cluster Configuration

AWS EKS

AWS Cloud Anomaly Detection for Kubernetes cluster monitoring involves four AWS components:

  • An EKS cluster
  • CloudWatch
  • Kinesis Firehose
  • S3 bucket

The EKS audit logs will be directly logged in CloudWatch where a Kinesis Firehose will stream the logs to an S3 bucket. A Cloud Anomaly Detection sensor will analyze the logs coming into the bucket and send anomalies to InsightCloudSec. InsightCloudSec provides a CFT to configure each component correctly, including roles, policies, the sensor, and an S3 bucket (if one does not exist already). Each cluster can have its own bucket or many clusters can share a bucket, but InsightCloudSec recommends placing the bucket in the same account as the cluster.

To configure AWS EKS cluster monitoring:

  1. In a browser window, login to the AWS console.
  2. Navigate to the CloudFormation service.
  3. Click Stacks in the navigation menu.
  4. Click Create Stack > With new resources (standard).
  5. Click Template is ready for the Prerequisite, then Amazon S3 URL for the Template source.
  6. Provide the following S3 URL: https://s3.amazonaws.com/get.divvycloud.com/prodserv/k8/s3-firehose/r7-ics-eks-s3-firehose.yaml
  7. Click Next.
  8. Specify the stack details:
    1. Provide a stack name.
    2. Update the Parameters as necessary.
      • By default, the CFT does not create a new S3 bucket for the audit logs. If you have an existing S3 bucket you would like to use, provide a Valid AWS Bucket Name
      • Provide the Account Number for the AWS Account that contains the S3 bucket.
      • Provide the EKS Cluster Name
  9. Select the I acknowledge that AWS CloudFormation might create IAM resources with custom names checkbox.
  10. Click Create Stack.
  11. Optionally, verify the CFT finishes creating the resources successfully.

Inside the monitored S3 bucket, EKS logs will begin to appear. They logs use the following folder structure: EKSLogs/<account-number>/<cluster-name>/<year>/<month>/<day>/<hour>. Regular Harvesting cadence will pick up any EKS clusters that are part of accounts that have been onboarded and they will be displayed on the Kubernetes Clusters tab under Cloud Anomaly Detection (Audit Log Monitoring) within the Cloud Accounts page. The last step is to configure storage for the log. Storage configuration enables you to update basic details for the bucket itself as well as the IAM Role Configuration associated with the bucket. See Storage Configuration: Kubernetes Cluster Monitoring for details.

Storage Configuration

Storage configuration enables you to update basic details for the audit log bucket itself as well as the IAM Role Configuration associated with the bucket. Storage configuration differs if you're working with organization and account monitoring or Kubernetes cluster monitoring.

Optional for Account Monitoring Users

This section is optional for account monitoring users as you only need to update the storage configuration if a different bucket or role was used over the default options. See AWS CloudTrail for more information.

Organization and Account Monitoring

  1. Login to InsightCloudSec as an Admin.

  2. Navigate to Cloud > Cloud Accounts > Audit Log Monitoring > Audit Logs.

  3. In the Bucket Name column, next to an entry for the CloudTrail bucket you just configured, click the Edit icon (Pencil).

  4. Update the Storage Configuration fields as necessary:

    • Resource Name is the name of the bucket that contains the audit logs
    • Region is the name of the region containing the bucket
    • Prefix is the directory name in the bucket that contains the audit log data (e.g., CloudTrail log location). The default prefix is AWSLogs/<account-number>.

  5. (Optional), update the bucket's IAM Role Config:

    Bucket Role Configuration

    If the role exists within InsightCloudSec already:

    1. click Existing IAM Role Config.
    2. Search for and select the role, then click OK.

    If the role does not exist within InsightCloudSec already:

    1. Click Create a new IAM Role Config.
    2. Provide the Role ARN for primary harvesting role to be assumed.
    3. Optionally, update the default Session Name.
    4. Select the the AWS Partition where the Role is located.
    5. Update the authentication dropdown as necessary.
      • For scenarios where the Harvesting role is in the same account as the CloudTrail bucket and/or encryption is used on the bucket: select Instance Profile
      • For scenarios where the Harvesting role is in a different account than the CloudTrail bucket: select IAM User via API Keys
        1. Click Create New Access Key Config, then copy/paste the Access Key and Secret Access Key you created earlier
    6. Click OK.

Kubernetes Cluster Monitoring

  1. Login to InsightCloudSec as an Admin.

  2. Navigate to Cloud > Cloud Accounts > Audit Log Monitoring > Kubernetes Clustesr.

  3. In the Bucket Name column, next to the entry for the cluster you just configured, click the Edit icon (Pencil).

  4. Update the Storage Configuration fields as necessary:

    • Resource Name is the name of the bucket that contains the EKS audit logs.
    • Region is the name of the region containing the bucket.
    • The Prefix is automatically determined and is not editable.

  5. (Optional), update the bucket's IAM Role Config:

    Bucket Role Configuration

    If the role exists within InsightCloudSec already:

    1. click Existing IAM Role Config.
    2. Search for and select the role, then click OK.

    If the role does not exist within InsightCloudSec already:

    1. Click Create a new IAM Role Config.
    2. Provide the Role ARN for primary harvesting role to be assumed.
    3. Optionally, update the default Session Name.
    4. Select the the AWS Partition where the Role is located.
    5. Update the authentication dropdown as necessary.
      • For scenarios where the Harvesting role is in the same account as the CloudTrail bucket and/or encryption is used on the bucket: select Instance Profile
      • For scenarios where the Harvesting role is in a different account than the CloudTrail bucket: select IAM User via API Keys
        1. Click Create New Access Key Config, then copy/paste the Access Key and Secret Access Key you created earlier
    6. Click OK.