S3 Archiving

InsightOps S3 Archiving allows InsightOps customers to utilize the inexpensive and massive storage capabilities of Amazon Web Services’ S3 Archiving features. S3 Archiving sends your InsightOps Logs to an AWS S3 Archive, (a bucket of your choice) for storing your log entries received in your InsightOps account as a backup.

This S3 Archiving functionality will occur once a day and store the entire day’s logs received the day before.

• Inexpensive and expansive storage of your InsightOps Data
• Longer retention beyond your InsightOps plan.
• Saves you money because these are not stored on InsightOps within your retention plan.
• Users may potentially analyze these logs in a manner in which InsightOps doesn’t provide at the moment.

You must have or create an S3 Bucket. The Rapid7 account must also have S3 permissions set on the specific S3 Bucket (Detailed Below)

S3 Archiving makes it possible for the logs that you forward to InsightOps to also be archived on AWS S3.

You simply need to provide list and upload permissions to an S3 bucket on your AWS account. If you do not have an AWS account and wish to archive your InsightOps logs onto S3, you can signup to AWS.

• Create an S3 bucket on AWS account where you wish to archive your InsightOps data.
• You can find your bucket names under S3 in the “All Buckets” view of your AWS console.
• The account that you need to give permissions to is a9c2e4259cad99e03b67c7450d6cc9c0d4f3243363c80ca73f6b5152ff293bb0
• This account requires list, view, and upload permissions
• This account will not be available to choose from the drop down; you will need to type in the account manually. If this account does not work for you in the verification process please contact Rapid7 Support who will manually add your bucket to the service.
• After entering the account, AWS may shorten this account name to “archive”

To enable archiving to an S3 bucket, after creating the S3 bucket in AWS as detailed above:

• Login to your InsightOps Account
• Go to your Account Settings in the left hand navigation
• Navigate to the account setting area and select the “S3 Archiving” Tab
• Click the "Enable S3 archiving" slider so it turns green
• Enter the exact name of your S3 Bucket that you created on your AWS account to store your InsightOps archived log data
• Click Save and the InsightOps will validate your settings. If everything is validated correctly, you will see a success banner

S3 Archiving is now set up and will run daily to back up your InsightOps logs to this S3 Archive.

Note: Archiving will not begin immediately. You should see logs in your S3 Archive approximately 24 hours after set up.

This S3 archive will store logs from this point on not retrospectively, so logs received by InsightOps from time of set up will be archived.

This step is only required if you want to enable server side KMS encryption on your bucket. If so, follow these steps:

1. Create a new KMS key:

   • Log on to your AWS Account.
   • Select KMS > Customer-managed keys > Create a key.
   • Select 'Symmetric' as your key type.
   • Add labels you would like attached to your KMS key.
   • Define key administrative permissions.
   • You can define key usage permissions, although this is not needed.
   • Review your existing key policy:
   • You should insert the following under your existing 'Statement':

 
1
{
2
            "Sid": "Allow use of the key by Rapid7",
3
            "Effect": "Allow",
4
            "Principal": {
5
                "AWS": "arn:aws:iam::029723416200:role/learchiving"
6
            },
7
            "Action": [
8
                "kms:Encrypt",
9
                "kms:Decrypt",
10
                "kms:ReEncrypt*",
11
                "kms:GenerateDataKey*",
12
                "kms:DescribeKey"
13
            ],
14
            "Resource": "*"
15
        }

2. Select S3 Bucket > Properties tab > Default Encryption and click 'Edit'.
3. Assign your newly created key to your bucket.