AWS Scan Engines

The AWS Scan Engine Amazon Machine Image (AMI) allows you to understand and manage risk associated with your dynamic EC2 assets. You can easily add one or more Scan Engines using the Rapid7 Scan Engine listing on the AWS Marketplace.

Once you create an AWS Dynamic Discovery connection, the Security Console pulls a list of EC2 instances using the AWS API. The engine scans these instances for vulnerabilities using port scans, service detection, and a combination of unauthenticated and authenticated checks.

Dependencies

• Before deploying a Scan Engine, you must deploy and license a Security Console.
• To use the AWS Scan Engine, you must configure a Dynamic Discovery connection with the AWS account(s) you want to scan. You must specify a Scan Engine to use when creating a Dynamic Discovery connection, so make sure you deploy your AWS Scan Engine first before you begin configuring your discovery connection.
• When launching an AWS Scan Engine, you must pair the engine to your Security Console. The AWS Scan Engine does not have a user-accessible interface, so you must provide console information to the engine when launching the AMI. This is done using EC2 user data.
• The Security Console must allow access on TCP port 40815 to receive data from the Scan Engine.
• The Scan Engine does not need to allow new inbound connections, but it must be able to establish outbound communication to port 40815 on the Security Console.
• For accurate scan results, the EC2 security group that the Scan Engine uses should allow unfiltered communication with all targets (such as the EC2 instances you want to scan) on all ports over their private IP addresses. Similarly, all target assets must belong to a security group or groups that allow the Scan Engine’s security group to communicate with the assets on all ports over their private IP addresses. If you deploy the Scan Engine using the CloudFormation template, you have the option of having CloudFormation automatically create properly configured security groups for your Scan Engine and targets. The setup of security groups will be discussed in more detail in the installation instructions.

Known Constraints

• The Scan Engine available on the AWS Marketplace is designed to only scan assets that have been detected by Dynamic Discovery. This is done to ensure you are in compliance with AWS policies on penetration testing and not unintentionally scanning assets you do not own.
• In order for a Scan Engine in AWS to scan across multiple VPCs, you must have your VPCs configured to allow traffic to flow between them. You can achieve this using several options, including VPC peering or Transit Gateway. VPC peering tends to be the most secure option, but you will need to decide what’s the best approach for your specific situation. Just like with on-premises network firewalls, if you do not want to facilitate communication between VPCs, you will need to install a Scan Engine in each VPC. You can also consider a different approach to assessment, like embedding the Insight Agent on all EC2 instances.
• The Security Console does not scan some smaller EC2 instances (such as m1.small or t1.micro). These instances may appear in Dynamic Discovery results, but they will not be scanned.

Installation, Configuration, and Use

Consult the following procedures to deploy an AWS Scan Engine in your environment.

Prepare the Security Console and your AWS environment

1. The Security Console needs to accept incoming connections on port 40815 from the Scan Engine.
   • If your Security Console is hosted in AWS on an EC2 instance, modify the instance's security group to allow this.
   • If your Security Console is hosted in another environment (such as an on-premises deployment), modify your firewall and any other applicable security systems to allow the appropriate incoming connections.
2. Have your Security Console IP address or hostname on hand. You will need to reference this information in the next section.
3. Generate a shared secret in the Security Console. To generate a shared secret, navigate to Administration > Scans > Manage scan engines. Generate a new shared secret at the bottom of the page.
   • Be aware that shared secrets are valid for only 60 minutes, so plan on completing your deployment within that time frame if you generate one now. If you prefer, you can skip this step until later and return to the Security Console to generate the shared secret when you need it.

Access the Rapid7 AWS Scan Engine on the AWS Marketplace

Now that your Security Console and AWS environment are prepared, you're ready to install the Scan Engine.

1. Sign in to the AWS account where you want to install a Scan Engine.

2. Go to the Rapid7 AWS Scan Engine listing in the AWS Marketplace.
3. Click Continue to Subscribe in the upper right corner of the page, then click Continue to Configuration.
4. Under Fulfillment Option, we recommend choosing CloudFormation Template as it automatically sets up the Scan Engine as well as the required EC2 security groups. If you prefer to configure these yourself, choose Amazon Machine Image.
5. Regardless of which option you choose, you will also need to select the software version (we always recommend the latest version) and the region where your Scan Engine should run. After you make these selections, click Continue to Launch in the upper right corner.

Both the CloudFormation Template and Amazon Machine Image procedures are detailed in the following sections.

Deploy an AWS Scan Engine using the Marketplace CloudFormation Template

This procedure details the CloudFormation Template deployment method continuing from step 4 of the previous section.

5. Under the Choose Action dropdown, select Launch CloudFormation, then click Launch. The Scan Engine template will open in CloudFormation. No configuration is required on the page that first appears, so click Next.
6. Enter a name for your stack.
7. In the Instance Type dropdown, you can change the type of EC2 instance that your Scan Engine will run on. The default selection is suitable for most situations.
8. Changing the size of the root disk from its 100GB default is not necessary, but you can adjust it using the Root Volume Size field if you want to.
9. Select the VPC and subnet where you want your Scan Engine to live. Make sure you select a subnet that is inside your selected VPC.
10. In the Assign a Public IP Address to Your Scan Engine field, select Yes if the Scan Engine will need a public IP address in order to connect with your InsightVM Security Console. If your Scan Engine is located on the same VPC as your Security Console, this should not be necessary.
11. Select whether or not you would like to have the stack create new EC2 security groups.
    • Selecting Yes will create one security group for your Scan Engine and another for the EC2 instances you want to scan. If you have not previously set up a Rapid7 Scan Engine in your AWS accounts, we recommend selecting Yes.
    • If you've previously created security groups to support a Rapid7 Scan Engine, then you can select No. In this case, you'll need to paste the ID of the security group you want your new Scan Engine to use into the Existing Scan Engine Security Group ID field.
12. If you are running your InsightVM Security Console in AWS and you have to set the Create New Security Groups field to Yes, then you must also set the Add Ingress to Console Security Group field to Yes and enter the ID of the Security Console's security group into the Console Security Group to Update field. This instructs the CloudFormation template to update the security group for your Security Console so that the Scan Engine can access it.
13. In the Security Console Host field, specify the IP address of your Security Console if it's not hosted in AWS. If your console is hosted in AWS, navigate to EC2, find the EC2 instance where the console is running, and copy the instance's Private DNS or Private IP address. Paste this address in the Security Console Host field.
14. Enter the Security Console shared secret you generated in the Security Console Secret field. If you have not generated your shared secret or if it’s been more than 60 minutes since you generated your secret, generate a new one before proceeding.
15. Click Next.
16. Apply optional items like tags to your stack according to your organization's best practices for AWS. Click Next.
17. Review your stack details and click Create Stack at the bottom of the page. The CREATE_COMPLETE status indicates that your stack has been created successfully.
18. After deployment, it can take up to 15 minutes for the Scan Engine to pair with the Security Console. Verify this pairing by checking your listed Scan Engines in Administration > Scans > Manage engine scans.

Before you try to use your new Scan Engine, make sure that the assets you would like to scan are members of a security group that your Scan Engine can access. If you elected to have the CloudFormation Template create new security groups for you, add your target assets to the newly created ScanTargetsSG security group. Otherwise, ensure that you add all target assets to a security group that allows all traffic on all ports from the security group containing your new Scan Engine. Check the Amazon Web Services article for more information on this process.

After you verify that everything is in the right security group, make sure you set up Dynamic Discovery before attempting to run a scan. See the Scan and review section below for more details.

Deploy an AWS Scan Engine using the Marketplace AMI

This procedure details the Marketplace AMI deployment method continuing from step 4 of the previous section.

5. Under the Choose Action dropdown, select Launch through EC2.
6. Choose an instance type. Rapid7 recommends m5.large or higher. Proceed to Configure Instance Details.
7. Specify the VPC and subnet where the Scan Engine should live.
8. Generate a shared secret in your Security Console if you have not done so already.
   • To generate a shared secret, navigate to Administration > Scans > Manage scan engines. Generate a new shared secret at the bottom of the page.
9. Expand Advanced Details at the bottom of the page and provide the following user data. Replace the portions in braces {} with information about your Security Console:

 
1
NEXPOSE_CONSOLE_HOST={hostname or ip of your console}
2
NEXPOSE_CONSOLE_PORT=40815
3
NEXPOSE_CONSOLE_SECRET={shared secret generated earlier}

10. Proceed to Add Storage. Set the root volume storage size. Rapid7 recommends at least 100GB.
11. Proceed to Add Tags. Add any tags that your Scan Engine EC2 instance should have according to your organization's best practices.
12. Proceed to Configure Security Group.
    • The Scan Engine's security group should have no inbound ingress rules that prevent all remote access over the network. The Scan Engine's outbound rules should allow outbound TCP access on port 40815 to facilitate communication with your Security Console.
    • The Scan Engine's security group should also allow outbound access on all protocols and ports to the security group or groups that control access to the assets that you intend to scan.
    • The security group or groups controlling access to the assets being scanned must also allow ingress on all protocol and ports from the Scan Engine's security group. Check the Amazon Web Services article for more information on this process.
13. Proceed to Review and Launch. Launch the instance.
14. After deployment, it can take up to 15 minutes for the Scan Engine to pair with the Security Console. Verify this pairing by checking your listed Scan Engines in Administration > Scans > Manage scan engines.

Scan and review

After you have your AWS Scan Engine set up, you can begin using it to scan your AWS assets. Complete the following procedure to configure the required site and dynamic discovery connection that you will need:

1. In your InsightVM Security Console, expand the Create dropdown in the upper left corner and click Site.
2. Give your new site a name.
3. Complete the site configuration by specifying scan credentials, a scan template, and a schedule that suits your needs, but do not configure anything on the Assets tab.
   • Your dynamic discovery connection will manage which assets get scanned, so leave the fields on this tab blank for now.
4. Save the site, but do not scan it yet.
5. Create a new AWS Asset Sync discovery connection. In the Consumption Settings section of the connection configuration page, select the site you just created.
6. Complete the rest of your discovery connection configuration and click Save.

Troubleshoot

See the following troubleshooting solutions if you encounter issues during your deployment.

Access Scan Engine logs

Although the AWS Scan Engine doesn’t allow interactive logins, you can check the AWS system log for errors related to the user data you provided. Follow these steps to view this log in your AWS console:

1. Navigate to the EC2 console.
2. View your instances and select the Scan Engine you created earlier.
3. Click Actions > Instance Settings > Get System Log.
4. Look for messages that start with pair-nexpose-engine.

If your Security Console is in AWS, verify that the console's security group allows access from the engine on port 40815. Verify that the Scan Engine's security group allows outbound access to the console's security group on port 40815.

If your Security Console is not in AWS, verify that the engine's security group allows outbound access to where the console is on port 40815.