Monitor an ongoing scan

When scanning your app, you may wish to monitor the ongoing scan to make sure there are no problems and that everything is working as expected. You can monitor the scan from the “Scan Status” window which launches as soon as you begin your scan. If there are errors or network problems, AppSpider will not be able to provide a comprehensive coverage report about your vulnerabilities, so it’s important to keep a lookout for issues.

Status

The Status tab has the following sections:

  • Summary
  • Per Attack
  • Operation Log
  • Traffic Log

Summary

The Scan summary window is the default screen that appears when the Scan Status window is launched. Summary information includes:

  • Current Domain - The target domain for this scan configuration. This is the URL that you added to the URL List in the “Main Settings” screen of the Scan Config wizard.

  • Form authentication - Form authentication is how AppSpider uses an attack module to log in to your application. This field will populate after authentication has happened and after AppSpider has used Regex to determine what kind of authentication occurred. The form authentication states are “Logged In,” “Not Logged In,” and “Not Logged Out.”

    • “Logged In” indicates an attack module was able to successfully log in. Successful attacks can include macros, simple form authentication, or simple forms.
    • “Not Logged In” indicates an attack module was not able to successfully log in to your application. Some attacks that fail include selenium or HTTP authentication.
    • “Not Logged Out” happens in rarer circumstances where the attack cannot find or utilize a logout feature, such as with HTTP Authentication or MTAL. This can happen if your application has a pop-up window for a login, but nothing to use for a logout.

  • Scan Status - This field can have the values “None”, “Scanning”, and “Completed”.

  • Start time - The start time for the scan as per the system clock.

  • Time Elapsed - Time elapsed in hours, minutes, and seconds since the start of the scan.

  • Time Remaining - An estimate of the remaining time for the scan based on the network speed and number of attacks remaining. If the application does not have enough information to make an estimate, the field will show the value “Unknown” or “00:00:00”.

Monitor the above areas for errors and potential issues, which may include problems with your form authentication, long delays between requests, or multiple failed requests. However, monitoring also includes taking note of operational information, such as:

Overall progress percentage

Includes not just the completion of the scan, but other activities of AppSpider, such as creating a report.
This metric is dynamic, and can change dramatically based on the number of linked discovered and crawled.

Scanning percentage bar

Displays the number of crawled URLs and how many attacks were used against those URLs.
This metric is dynamic and can change dramatically based on the number of links discovered and crawled.

Findings Summary

Provides metrics on number of vulnerabilities found per class, along with a visual representation.

Network Monitor

Provides network data in regards to your scan, such as the number of requests made and response time.

Events table

Lists the history of actions taken during the scan, such as when you pause and resume a scan, or when AppSpider performs actions like “Starting Second Stage of Scanning,” along with time and severity of these historical events.

Logging

You have the option to enable both Operation Logs and Traffic Logs before and during an AppSpider scan. To configure logging before a scan starts:

  1. In the top navigation tabs, select ** Configurations > New Scan Config.** A new screen will appear.
  2. Select the Performance section.
  3. In the “Logging Options” section, click to enable or disable your logging option.

To configure logging during a scan:

  1. Go to the Status tab for your ongoing scan.
  2. Select either Operation Log or Traffic Log.
  3. At the top of the logging table, select the Logging Enabled button to enable it. The button should turn blue, and traffic should appear.

Note

Rapid7 recommends not enabling Traffic Logs unless you are specifically troubleshooting for an issue.

Note

Rapid7 recommends enabling Operation Log to view any problems that occur during start up.

Operation Log

If you enable the operation log, it lists all of the events that occur and provides more detailed information about each event that are considered during scanning, such as:

  • Memory usage
  • Vulnerabilities identified
  • Attempted attacks, such as macros or authentication types
  • Initial links crawled
  • Free space in GB
  • Scheduled resources
  • Proxy usage

You can access the Operational log on the left side of AppSpider under the Status page. This log provides the index number of each event, what time the event occurred, the severity of the event, and the message associated with the event and index ID, such as “Running attack ‘URC4[0]’, module ‘Unvalidated Redirect’ on ‘Parameter: ‘submit_login’, location ‘Post’, link ‘http://www.yourappurl.com.”

Traffic Log

The traffic log captures every single request and response happening between your application and AppSpider in real time. Because this can produce massive amounts of data, it is best practice to enable the traffic log only when you are troubleshooting a specific issue. You can access the traffic log on the left side of AppSpider under the Status page.

The traffic log displays information such as response codes, methods of attack, and URLs. You’ll also see information about individual requests and responses pertaining to each traffic event log.

Traffic logs also show the events happening in the Operation log, such as the attack policy and module used for the attack.