Distributed Scan Engines

Distributed Scan Engines are separate from the Security Console and are strategically provisioned and located in a way that makes your scanning environment as efficient as possible. If you intend to maintain a production deployment of the Security Console, distributed Scan Engines are an absolute necessity.

This article guides you through the deployment and configuration process for distributed Scan Engines.

Deployment Process at a Glance

Deploying a distributed Scan Engine involves the following procedures:

• Installing a new Scan Engine on a separate host machine
• Pairing the Scan Engine to the Security Console according to your communication method of choice
• Refreshing the Scan Engine in the Security Console to verify that the pairing was successful

Methods of Communication

Scan Engines and Security Consoles must be able to communicate with each other in order to initiate scans and integrate scan data. Distributed Scan Engines can communicate with a Security Console in two ways:

• Console-to-Engine - This is the standard method of communication. In this case, the Security Console directs the Scan Engine to perform a task by initiating a TCP connection over port 40814.
• Engine-to-Console - Also known as the Reverse communication method, engine-to-console pairings rely on the Scan Engine initiating the connection to the Security Console when the engine starts.

You can read more about these configurations on the Scan Engine Communication Methods Help page.

Requirements

To ensure that you’re set up for success with a distributed Scan Engine, make sure you complete the following preliminary steps:

• Verify that your intended Scan Engine host meets the system requirements.
• If firewalls are present on your network, make sure you allowlist the necessary ports for your Security Console and Scan Engine host according to the communication method of your choice.
• Configuration for firewall ports depends on whether the scan engines are inside or outside of your network.
  • Inside the network - Any internal scan engine, an engine that is located within your network and running authenticated scans against internal assets, should have all ports open between it and any assets being scanned, so that any closed ports are due to the asset not advertising them instead of firewall limitations.
  • Outside the network - We recommend allowlisting the Scan Engine for any IDS that could potentially block the Scan Engine from connecting to assets.

Consult the following table for the default port allowlist requirements.

Download and Install the Scan Engine

After verifying that you meet the hardware and networking requirements, you’re ready to download and install the Scan Engine on your intended host machine.

Download the Scan Engine Installer

The Nexpose product installer is also responsible for installing distributed Scan Engines. Visit the download section of the quick start guide to download the Linux or Windows installer according to the operating system of your intended host machine.

Install the Scan Engine

Launch the product installer to get started. Follow the initial prompts until you reach the component selection and communication direction step.

Select Components

After going through the necessary acknowledgements, you’ll be prompted to select which components you want to install.

Select Scan Engine only. This tells the installer that you intend to deploy a distributed Scan Engine.

Select a Communication Direction

After selecting your components, you’ll be prompted to select a communication direction.

If you select the Engine-to-Console method, you will have the opportunity to configure a reverse pair with your Security Console during the Scan Engine installation. Although you can skip this pairing step if you want to, Rapid7 recommends that you take advantage of this pairing opportunity since the post-install reverse pairing procedure involves more complicated steps.

To configure a reverse pair during a Scan Engine installation:

1. You can choose to pair the Scan Engine with the Insight Platform. This is optional and not required to complete pairing with the Security Console.
2. Select Yes to pair the Scan Engine with the Security Console.
3. When prompted by the install wizard, enter the IP address for the Security Console. Then, proceed with the default settings for Console TCP Port [40815].
4. Provide the installer with the Security Console shared secret.
   • You can generate a shared secret in the Security Console. On the Administration tab, in the Scans section, click Manage scan engines. In the Generate Scan Engine Shared Secret section, click Generate.

5. Test your connection to ensure that your Security Console and Scan Engine can communicate properly.
6. Continue with the rest of the Scan Engine installation.
7. When the install wizard completes you will be prompted to start the Scan Engine service. This may take a few minutes for the Scan Engine service to finish loading.
   • Linux - systemctl start nexposeengine.service
8. After your Scan Engine finishes installing, proceed directly to the Refresh Your New Scan Engine section of this guide.

If you select the Console-to-Engine method, you’ll need to configure a standard pair with your Security Console after the Scan Engine installation completes. Continue with the rest of the installation at this time. After your Scan Engine finishes installing, proceed to the Pair Your Scan Engine to the Security Console section of this guide.

Pair Your Scan Engine to the Security Console

All new Scan Engines must be paired to the Security Console in order to be usable for scanning. These engine pairing procedures differ based on the method of communication you want to implement.

Consult one of the following pairing procedures for your communication method of choice:

• Standard Pair (Console-to-Engine)
• Reverse Pair (Engine-to-Console)

Standard Pair (Console-to-Engine)

In order to configure a console-to-engine pairing, the Security Console must be made aware that a new Scan Engine is available for use and must be provided with instructions on how to reach it. Consequently, the first step of all standard pairing procedures is to add your new Scan Engine to the Security Console.

You can add a new Scan Engine within a site configuration or through the Administration page.

Add an Engine in a Site Configuration

To add a Scan Engine in a site configuration:

1. In a new or existing site configuration, click the Engines tab.
   • You can create a new site by expanding the Create dropdown next to the left navigation menu, or access an existing site by clicking its edit icon in the “Sites” table of the Home page.
2. Click the Add Scan Engine subtab.
3. Name your Scan Engine.
4. Enter the IP address of your Scan Engine in the Address field.
5. If you need to adjust the default port number, modify the value shown in the Port field.
6. Assuming your site has the minimum set of required information, click Save when finished.

Add an Engine through Administration

To add a Scan Engine through the Administration tab:

1. On the Administration page, in the Scans > Scan Engines section, click Manage scan engines.
2. Click New Engine.
3. On the General tab, name your Scan Engine.
4. Enter the IP address of your Scan Engine in the “Address” field.
5. Click Save when finished.

Properly added Scan Engines generate a consoles.xml file on the Scan Engine host. You will modify this file in the next step.

Modify consoles.xml

The consoles.xml file generated on your Scan Engine host in the previous step contains an entry for the Security Console that added the Scan Engine. You must enable the console to complete the pairing.

To modify the consoles.xml file for a Linux or Windows host:

1. Browse to the consoles.xml file in your Scan Engine directory. By default, this file is located in the following places according to the operating system of your Scan Engine host:
   • Linux - /opt/rapid7/nexpose/nse/conf/consoles.xml
   • Windows - Files\Rapid7\NeXpose\nse\conf\consoles.xml
2. Open consoles.xml with a text editor of your choice. Navigate to the <consoles> element. The Security Console that added the Scan Engine appears as a <console> element with several attributes.
   • You can identify the correct Security Console by checking that the lastAddress attribute matches the IP address of the Security Console you want to pair with.
3. Change the value for the enabled attribute from 0 to 1.
4. Save and close the consoles.xml file.
5. Restart the Scan Engine host so your changes can take effect.

Reverse Pair (Engine-to-Console)

If you took advantage of the reverse pairing configuration opportunity during your Scan Engine installation, then you’ve already completed this step! Proceed directly to the Refresh Your New Scan Engine section of this guide to verify that your Scan Engine is ready for use.

However, if you installed a Scan Engine with the Engine-to-Console method selected without completing the reverse pairing step, you must complete the pairing with a separate procedure. See the Post-Installation Engine-to-Console Pairing page for instructions on how to do this.

Refresh Your New Scan Engine

After completing a standard or reverse pair for your Scan Engine, you must refresh its status to verify that the Security Console can communicate with it properly.

To refresh your new Scan Engine in the Security Console:

1. On the Administration page, click Scans > Engines.
2. In the Scan Engines section, click Refresh Displayed Engines. This ensures that your Scan Engine table is up-to-date.
3. Locate the distributed Scan Engine that you paired to the Security Console. Click the icon in the “Refresh” column to complete the verification process.

If you have properly configured and paired your Scan Engine, it now displays up-to-date version and communication status information. The “Communication Status” column itself indicates both the current communication method by arrow and connection state by color. Arrows pointing to “Engine” indicate a standard pairing, while arrows pointing to “Console” indicate reverse pairing. Additionally, arrow icons can have the following color codes:

• Green - Scan Engine is active
• Orange - Scan Engine status is unknown
  • An “unknown” status indicates that the Security Console and the Scan Engine could not communicate even though no error was recorded. This is often the result of a significant lapse between pings. Refresh the Scan Engine status to attempt communication again.
• Red - There are three possibilities associated with this color code:
  • Pending Authorization - This state indicates that the Security Console has not yet been enabled on the Scan Engine. Consoles must be enabled on the Scan Engine during the pairing procedure.
  • Incompatible - This state indicates that the Security Console and Scan Engine are on different versions. Consoles and engines must be on the same version in order to communicate properly.
  • Down - This state indicates that the Scan Engine is offline.