AWS Cloud Trail

Amazon Web Services, or AWS, is a cloud service integration that allows you to track how your corporate cloud services are being used.

Required Information

In order to connect your AWS Cloud Trail to InsightOps, you'll need the following information beforehand:

  • Credential
  • Secret Key
  • S3 Bucket Name
  • S3 Key Prefix
  • Bucket Region Name
  • Refresh Rate (Minutes)

To gather this information, follow the steps below for configuration.

Before You Begin

Depending on the region your CloudTrail logs are stored in, the collector will need to be able to reach the following URL to collect the logs:

S3 Region

URL

US_STANDARD

https://s3.amazonaws.com

US_WEST_OREGON

https://s3-us-west-2.amazonaws.com

US_WEST_N_CALIFORNIA

https://s3-us-west-1.amazonaws.com

EU_IRELAND

https://s3-eu-west-1.amazonaws.com

EU_FRANKFURT

https://s3.eu-central-1.amazonaws.com

AP_SINGAPORE

https://s3-ap-southeast-1.amazonaws.com

AP_SYDNEY

https://s3-ap-southeast-2.amazonaws.com

AP_TOKYO

https://s3-ap-northeast-1.amazonaws.com

SA_SAO_PAULO

https://s3-sa-east-1.amazonaws.com

Enable CloudTrails in all regions

To get maximum coverage of CloudTrails monitoring, you should enable CloudTrails in all regions, even if you don't have any EC2 instances or other AWS resources running in all regions. This helps to ensure that, going forward, if an attacker compromises a resource in your AWS account that allows them to create/modify resources in other regions, you'll be able to monitor and alert on that behavior.

  1. In the AWS Console, go to CloudTrails → Trails → Add new trail
  2. Add a name for your trail in "Trail name"
  3. Select Yes for the "Apply trail to all regions" option
  4. Select Yes for the "Create a new S3 bucket" option
  5. Add a name for your S3 bucket. Record this for future steps.

Create IAM Policy

  1. In the AWS Console, go to IAM → Policies → Create Policy → Create Your Own Policy
  2. Add a Name and Description for your Policy. Record the name somewhere - you'll need it for later
  3. Use the following Policy template, which is based on the principal of least privilege and only allows access to the specific S3 bucket you created for your CloudTrails logs:
text
1
{
2
"Version": "2012-10-17",
3
"Statement": [
4
{
5
"Effect": "Allow",
6
"Action": [
7
"s3:Get*",
8
"s3:List*"
9
],
10
"Resource": [
11
"arn:aws:s3:::CloudTrailsS3BucketNameGoesHere",
12
"arn:aws:s3:::CloudTrailsS3BucketNameGoesHere/*"
13
]
14
}
15
]
16
}

Create IAM Group

  1. In the AWS Console, go to IAM → Groups → Create New Group
  2. Create a Group Name and select Next Step
  3. Select the IAM Policy you created earlier and select Next Step

Create and configure IAM User

  1. In the AWS Console, go to IAM → Groups → Add user
  2. Add a User name and select Programmatic Access under the "Access Type" section and select Next: Permissions
  3. Select the Group you created earlier and select Next: Review
  4. On the "Complete" page, select Show on the "Secret Access Key." Copy and save this User's Access Key and Secret Key in a secure location. You'll need it when setting up the CloudTrails Event Source in InsightOps.

Setup S3 bucket policy

  1. Find the ARN for the user associated with the access key configured in the collector:
  1. Find the bucket configured for the CloudTrail logs:
  2. Go to the bucket properties in S3 and click on Edit Bucket Policy:
  3. Add List* and GetObject rights to the bucket that match the ARN of the user
text
1
{
2
"Version": "2012-10-17",
3
"Statement": [
4
{
5
"Sid": "AWSCloudTrailAclCheck20150319",
6
"Effect": "Allow",
7
"Principal": {
8
"Service": "cloudtrail.amazonaws.com"
9
},
10
"Action": "s3:GetBucketAcl",
11
"Resource": "arn:aws:s3:::CLOUDTRAILS S3 BUCKET NAME"
12
},
13
{
14
"Sid": "AWSCloudTrailWrite20150319",
15
"Effect": "Allow",
16
"Principal": {
17
"Service": "cloudtrail.amazonaws.com"
18
},
19
"Action": "s3:PutObject",
20
"Resource": "arn:aws:s3:::CLOUDTRAILS S3 BUCKET NAME/AWSLogs/AWS ACCOUNT NUMBER/*",
21
"Condition": {
22
"StringEquals": {
23
"s3:x-amz-acl": "bucket-owner-full-control"
24
}
25
}
26
},
27
{
28
"Sid": "",
29
"Effect": "Allow",
30
"Principal": {
31
"AWS": "arn:aws:iam::AWS ACCOUNT NUMBER:user/IAM USER NAME"
32
},
33
"Action": "s3:List*",
34
"Resource": [
35
"arn:aws:s3:::CLOUDTRAILS S3 BUCKET NAME",
36
"arn:aws:s3:::CLOUDTRAILS S3 BUCKET NAME/*"
37
]
38
},
39
{
40
"Sid": "",
41
"Effect": "Allow",
42
"Principal": {
43
"AWS": "arn:aws:iam::AWS ACCOUNT NUMBER:user/CLOUDTRAILS S3 BUCKET NAME"
44
},
45
"Action": "s3:GetObject",
46
"Resource": "arn:aws:s3:::CLOUDTRAILS S3 BUCKET NAME/*"
47
}
48
]
49
}

How to Configure This Event Source

  1. Click the "Add data" button in the top navigation
  2. Select the Cloud Service icon from the Security Data section
  3. Select your collector, and optionally name your event source
  4. From the list of event source options, choose AWS CloudTrail
  5. Optionally choose to send unfiltered logs
  6. Select your existing Credentials or create a new one.
  7. Enter the Secret Key created in previous steps.
  8. Enter the S3 Bucket Name created in previous steps.
  9. Enter the S3 Key Prefix created in previous steps.
  10. Select the Bucket Region Name.
  11. Enter the refresh rate in minutes.
  12. Configure any Advanced Event Source settings.

Common Issues / Troubleshooting

If you find that InsightOps is not ingesting logs and data is not appearing, please do the following:

  1. Check that your IAM policy is correct
  2. Check that you've used the right region
  3. Ensure there are actually logs in the S3 bucket