Configure Scan Authentication

Modern web applications have more complex interfaces than simple HTML websites. Automated Login allows you to automatically authenticate to modern apps in a scan without relying on a macro. During a scan with Automated Login authentication, InsightAppSec analyzes and identifies the login pages, enters the credentials, and logs in to the app automatically. As well as this, it identifies when the app has been logged out and logs back in, preventing authentication loops.

Step 1: Configure Automated Login

1. On the Apps page, select your app.
2. On the Scan Configs tab, open your scan config.
3. On the Authentication > Site Authentication page, select Automated Login.
4. Enter your username and password in the respective fields.
5. (Optional) Test the login credentials.
6. Save the Scan Config.
   • Click Save to save and close.
   • Click Save and Scan to save and run the scan.

Test your login credentials

Test the login credentials while adding or updating a scan config to catch incorrect credentials and resolve them before running into an issue during a scan.

1. In the scan config, on the Authentication tab, go to Site Authentication > Automated Login.
2. Enter the username and password. In the Verify Credentials section, enter the Login URL
3. Click Verify. A window appears and shows the realtime login attempt.
4. After the credentials are successfully verified, save the scan config.
   • Click Save to save and close.
   • Click Save and Scan to save and run the scan.
     When you save the scan config, the credentials are encrypted. To verify credentials on a saved scan config, re-enter the username and password.

Step 2: Configure a check to test whether the scan is authenticated and logged in

Occasionally, the scan engine can trigger a logout action that means it cannot continue to scan areas of the website that require authentication. To mitigate this happening, the scan engine constantly checks the authentication status. One method the scan engine employs to achieve this is by checking the location field of the header response, to identify a login redirect. To configure this feature for your website, perform the following steps:

1. In the first field, enter a URL from your site that can only be reached if you are logged in.
2. Press View Logged Out Response.
3. In the second field, the location string automatically populates. This is the string the scan engine looks for in the Location field of the response header to identify a login redirect. This field accepts regex, which you can use to anonymize the string, for example if there is any user-specific element.

In an instance where the engine detects it has logged out, it then attempts to log in again.

InsightAppSec may sometimes be unable to reach the login page of your application, or the login form may become available only after a certain specific sequence of actions has been carried out on your website. For example, the login form may appear in a pop-up that gets dynamically generated via Javascript when the "Login" button is pressed from the "Administration" window. You can enable InsightAppSec to perform this sequence of steps by recording a macro. A macro is a sequence of actions, such as clicking of buttons, or text entry in a web page recorded in a .rec file. During the scan, InsightAppSec can replay the actions in this file to log in to the web application.

Step 1: Configure Macro Authentication

1. Open the Authentication > Site Authentication page and select Macro Authentication.
2. Click the Record New Macro button and enter the login URL for your application. Once you have done so click the Start Recording button.
3. A confirmation dialog will appear, notifying that the recording sequence has begun. The macro window will open at the URL you provided. Simply log in normally. Once you have logged in successfully, close the macro window and click the Stop Recording button.
4. A message will appear confirming that the recording was successful. Name your macro and click the Save Macro button. You should now see your recorded macro. [/block]
5. Save and run your config. It will now attempt authentication using your recorded macro. If successful, the following will appear in the scan’s event log:

Macros recorded by the AppSec plugin are set to the following by default:

 
1

2
  format: "xml"
3
  min_duration: 3000,
4
  element_path: "fullpath"
5
  event_type: "javascript"

Step 2: Configure a check to test whether the scan is authenticated and logged in

Occasionally, the scan engine can trigger a logout action that means it cannot continue to scan areas of the website that require authentication. To mitigate this happening, the scan engine constantly checks the authentication status. One method the scan engine employs to achieve this is by checking the location field of the header response, to identify a login redirect. To configure this feature for your website, perform the following steps:

1. In the first field, enter a URL from your site that can only be reached if you are logged in.
2. Press View Logged Out Response.
3. In the second field, the location string automatically populates. This is the string the scan engine looks for in the Location field of the response header to identify a login redirect. This field accepts regex, which you can use to anonymize the string, for example if there is any user-specific element.

In an instance where the engine detects it has logged out, it then attempts to log in again.

You may run into web applications built with technologies that are not supported by the InsightAppSec crawler. You can authenticate into such applications by using a web proxy tool such as the Traffic Recorder in the Rapid7 AppSec Toolkit. Using the proxy tool, you can record the interactions (e.g. HTTP GET and POST requests) between the front end application and the back end server in a Traffic File. InsightAppSec can replay these interactions to authenticate into your application.

Step 1: Authenticate with a Traffic file

1. Complete the steps for logging into your application and record the interactions in a traffic file on your computer. Traffic files can be of the following formats:
   • AppSec toolkit Traffic Files (*.trec)
   • Burp Files (*.xml)
   • Paros Files (*.txt)
   • WebScarab Files (conversationlog)
   • HAR (HTTP Archive) Files (*.har)
   • Fiddler Files (*.saz)
2. Open the Authentication > Site Authentication page and select Traffic.
3. Click the Choose File button.
4. Click the Upload File button and upload the traffic file from your computer.
5. Select the newly uploaded file or an existing traffic file from the "All my Files" tab in the popup.
6. Click the Use 1 Selected File(s) button.

Step 2: Configure a check to test whether the scan is authenticated and logged in

Occasionally, the scan engine can trigger a logout action that means it cannot continue to scan areas of the website that require authentication. To mitigate this happening, the scan engine constantly checks the authentication status. One method the scan engine employs to achieve this is by checking the location field of the header response, to identify a login redirect. To configure this feature for your website, perform the following steps:

1. In the first field, enter a URL from your site that can only be reached if you are logged in.
2. Press View Logged Out Response.
3. In the second field, the location string automatically populates. This is the string the scan engine looks for in the Location field of the response header to identify a login redirect. This field accepts regex, which you can use to anonymize the string, for example if there is any user-specific element.

In an instance where the engine detects it has logged out, it then attempts to log in again.

Many modern web applications use security measures, like two-factor authentication and CAPTCHA, that require manual intervention for logging in to the application. For example, your application may require a one time password sent via SMS. Multi-factor authentication will pause the active scan and allow you to interactively log in to the target application. After authentication is complete, you can resume the scan. The scan engine will continue to crawl and attack your application.

In order to enable multi-factor authentication, you’ll need:

• Chrome browser
• The latest version of the AppSec Chrome Extension
• Engine version number 7.2.049 or later, if you are using an on-premises scan engine

Step 1: Enabling Multi-factor Authentication for a Scan

To enable multi-factor authentication for your scan, go to Authentication > Authentication Type. Click the Authentication Type dropdown and select Multi-factor.

After you have started the scan:

1. Monitor the scans table. When your scan is ready for authentication, the “Status” column will show the message “Awaiting Authentication”.
2. Click on the name of the scan to open the scan window. You will see the “Authenticate Scan” button on the upper right side of the screen.
3. Click the Authenticate Scan button. If you have not installed the AppSec Chrome Extension, you will see a message prompting you to install it. If you have installed the extension, you will see the “Multi-factor Authentication” screen.
4. Enter the URL of the application and click Start. A popup message with the name of the file where the multi-factor authentication web traffic will be recorded on the Insight Platform will appear. Click OK to continue. A Chrome browser window will pop up and open the web page for your URL.
5. Log in to the application. When the login process is successful, close the Chrome window and return to the “Multi-factor Authentication” screen.

6. Click Stop. The Multi-Factor Authentication screen will close, and you will return to the scan window.
7. Monitor the “Event Log” to ensure that authentication has been successful. The Chrome extension records the web traffic of your authentication session and converts it into an HTTP Archive (HAR) file. The scan engine replays this traffic in memory to log in to your application. If the login is successful, you will see the message “Session ‘Authentication with Multi-Factor File:’ logged in” in the event log. You can also see the status “Logged In” under the Crawled Links count on the scan page.

If the login is not successful, you should ensure that the regular expression in the “Logged-in Regex” field from the “Advanced” screen matches the text in the web page that appears after authentication. If the indicator for logged-in state is hidden under a menu item, expand it so that InsightAppSec can examine it and check the logged-in state.

For example, if you want to use the presence of the “Logout” link as an indicator that user is logged in, expand the drop down where the Logout link is hidden as part of the authentication steps.

Step 2: Configure a check to test whether the scan is authenticated and logged in

Occasionally, the scan engine can trigger a logout action that means it cannot continue to scan areas of the website that require authentication. To mitigate this happening, the scan engine constantly checks the authentication status. One method the scan engine employs to achieve this is by checking the location field of the header response, to identify a login redirect. To configure this feature for your website, perform the following steps:

1. In the first field, enter a URL from your site that can only be reached if you are logged in.
2. Press View Logged Out Response.
3. In the second field, the location string automatically populates. This is the string the scan engine looks for in the Location field of the response header to identify a login redirect. This field accepts regex, which you can use to anonymize the string, for example if there is any user-specific element.

In an instance where the engine detects it has logged out, it then attempts to log in again.

Selenium is a framework for automated testing of web applications. Users can record actions like entering data in forms and clicking buttons using Selenium and replay them on demand to ensure that the web application behaves as desired.

InsightAppSec supports authentication using Selenium files, so you can record the actions needed to log in to your application in a Selenium .side file. During the scan, InsightAppSec can replay the actions in this file to log in to the web application. The following is a Selenium authentication file for Hackazon.

 
1
{
2
  "id": "21835ec2-a3fb-4be5-87e0-0008d439e4bd",
3
  "version": "2.0",
4
  "name": "Hackazon_Login",
5
  "url": "https://hackazon.webscantest.com",
6
  "tests": [{
7
    "id": "3c20000d-6927-42b5-b833-abc8bbf95fc8",
8
    "name": "Login",
9
    "commands": [{
10
      "id": "a682f8fe-5c36-4656-b6cd-16cc9de97024",
11
      "comment": "",
12
      "command": "open",
13
      "target": "/user/login",
14
      "targets": [],
15
      "value": ""
16
    }, {
17
      "id": "197ef686-b02b-4f4f-8b2f-c2e0caacbdd4",
18
      "comment": "",
19
      "command": "setWindowSize",
20
      "target": "1490x614",
21
      "targets": [],
22
      "value": ""
23
    }, {
24
      "id": "7f523e30-859d-4332-b1e0-43db5d53b755",
25
      "comment": "",
26
      "command": "click",
27
      "target": "id=username",
28
      "targets": [
29
        ["id=username", "id"],
30
        ["name=username", "name"],
31
        ["css=.form-group:nth-child(1) > #username", "css:finder"],
32
        ["xpath=//input[@id='username']", "xpath:attributes"],
33
        ["xpath=//form[@id='loginPageForm']/div/div/div/input", "xpath:idRelative"],
34
        ["xpath=//div/div/div/input", "xpath:position"]
35
      ],
36
      "value": ""
37
    }, {
38
      "id": "d3658c7f-04a6-4321-9fe0-ae27a10c4833",
39
      "comment": "",
40
      "command": "type",
41
      "target": "id=username",
42
      "targets": [
43
        ["id=username", "id"],
44
        ["name=username", "name"],
45
        ["css=.has-success > #username", "css:finder"],
46
        ["xpath=//input[@id='username']", "xpath:attributes"],
47
        ["xpath=//form[@id='loginPageForm']/div/div/div/input", "xpath:idRelative"],
48
        ["xpath=//div/div/div/input", "xpath:position"]
49
      ],
50
      "value": "test_user"
51
    }, {
52
      "id": "58efc617-ea4a-489e-bad3-751d46b32ee9",
53
      "comment": "",
54
      "command": "type",
55
      "target": "id=password",
56
      "targets": [
57
        ["id=password", "id"],
58
        ["name=password", "name"],
59
        ["css=.has-success > #password", "css:finder"],
60
        ["xpath=//input[@id='password']", "xpath:attributes"],
61
        ["xpath=//form[@id='loginPageForm']/div[2]/div/div/input", "xpath:idRelative"],
62
        ["xpath=//div[2]/div/div/input", "xpath:position"]
63
      ],
64
      "value": "123456"
65
    }, {
66
      "id": "f087221e-8b40-4635-bf7d-02d9fec8529f",
67
      "comment": "",
68
      "command": "click",
69
      "target": "id=loginbtn",
70
      "targets": [
71
        ["id=loginbtn", "id"],
72
        ["css=.row:nth-child(7) #loginbtn", "css:finder"],
73
        ["xpath=//button[@id='loginbtn']", "xpath:attributes"],
74
        ["xpath=//form[@id='loginPageForm']/div[3]/div/button", "xpath:idRelative"],
75
        ["xpath=//div[3]/div/button", "xpath:position"],
76
        ["xpath=//button[contains(.,'Sign In')]", "xpath:innerText"]
77
      ],
78
      "value": ""
79
    }]
80
  }],
81
  "suites": [{
82
    "id": "0674c923-d54a-4893-b9bc-47c8e630905b",
83
    "name": "Default Suite",
84
    "persistSession": false,
85
    "parallel": false,
86
    "timeout": 300,
87
    "tests": ["3c20000d-6927-42b5-b833-abc8bbf95fc8"]
88
  }],
89
  "urls": ["https://hackazon.webscantest.com/"],
90
  "plugins": []
91
}

Authenticate with a Selenium file

1. Complete the steps for logging in to your application and record the interactions in a Selenium file of the .side format.
2. On the Authentication > Site Authentication page, select Selenium.
3. Click the Choose File button.
4. Click the Upload File button and upload the Selenium file from your computer.
5. Select the newly uploaded file or an existing Selenium file from the "All my Files" tab in the popup.
6. Click the Use 1 Selected File(s) button.

Step 2: Configure a check to test whether the scan is authenticated and logged in

Occasionally, the scan engine can trigger a logout action that means it cannot continue to scan areas of the website that require authentication. To mitigate this happening, the scan engine constantly checks the authentication status. One method the scan engine employs to achieve this is by checking the location field of the header response, to identify a login redirect. To configure this feature for your website, perform the following steps:

1. In the first field, enter a URL from your site that can only be reached if you are logged in.
2. Press View Logged Out Response.
3. In the second field, the location string automatically populates. This is the string the scan engine looks for in the Location field of the response header to identify a login redirect. This field accepts regex, which you can use to anonymize the string, for example if there is any user-specific element.

In an instance where the engine detects it has logged out, it then attempts to log in again.

If your web application uses session cookies for maintaining a logged-in state, you can capture this cookie and use the Session Hijacking method for authentication. Copy and paste the session cookie into the available box.

The HTTP protocol supports authentication using a username and password. You can use this reference article to learn more about HTTP Authentication: https://docs.microsoft.com/en-us/dotnet/framework/wcf/feature-details/understanding-http-authentication

InsightAppSec supports the Basic, NTLM, and Kerberos protocols for HTTP authentication.

Configure the authentication process

1. Open the Authentication > HTTP Authentication screen.
2. Click the Enable HTTP Auth switch to enable HTTP authentication.
3. Enter your username and password in the respective fields.

HMAC is a MAC (Message Authentication Code) that uses a cryptographic hash function and a secret key to authenticate. To learn more about HMAC, see https://en.wikipedia.org/wiki/HMAC.

Configure HMAC authentication

1. Go to the Authentication > HMAC page.
2. Use the toggle button to Enable HMAC.
3. Enter the following information:
   • Username - The username for your hash algorithm.
   • Secret Key - A unique code used to compute the HMAC. It is known by both the sender and receiver.
   • Hash Algorithm - Choose the hash algorithm type.
     • Md5
     • Sha1
     • Sha256

• Header Generation DLL Filename - A user created DLL file (https://support.microsoft.com/en-us/help/815065/what-is-a-dll), which contains the cryptographic hash function. This is only needed if using an on-premise engine. If you don’t provide a file, the system will use the DLL provided by the engine.

You may need to authenticate your application on the client side so you can scan pages found behind your SSL. This authentication method can be used with both cloud and on-premise scan engines.

Configure SSL authentication

1. Click Choose File. You can click Upload File or drag and drop your SSL .pfx file.
2. Click Use Selected File
3. Optional: If a password is required, enter it in the provided box.
4. When you have successfully uploaded your .pfx file, save the scan config.
   • Click Save to save and close.
   • Click Save and Scan to save and run the scan.

OAuth (https://oauth.net/) is an authorization method that is used by applications to grant fine grained access to clients. InsightAppSec supports OAuth 2.0 which is the industry-standard protocol for authorization. OAuth 2.0 focuses on client developer simplicity while providing specific authorization flows for web applications, desktop applications, mobile phones, and living room devices. If your application has granted InsightAppSec the access to certain capabilities, you can enter the required details in the Authentication > OAuth screen. When starting a scan, InsightAppSec can provide these details to your application and receive an access token.

Set Up OAuth Authentication

If you want to scan an application that uses OAuth, you will need to know the grant type used by your application. You can usually get this information from your application developers. If you examine the traffic from a connection, you can also often see the grant type in the URL.

OAuth and Automated Login

The Authentication > OAuth screen has a number of options to configure your application's OAuth properties. You will need to click the Enable OAUTH switch to enable OAuth authentication, and then provide the required values based on your grant type.

• Resource Owner URL - An entity capable of granting access to a protected resource. Optional, in most cases should be equal to Resource Server URL
• Resource Server URL - The identifier for your API server. The resource server handles authenticated requests after the application has obtained an access token
• Authentication Server URL - The authentication server URL, obtained from your identity provider (reference: https://en.wikipedia.org/wiki/Identity_provider)
• Redirect URI - The URL that the authorization server will redirect the user back to, with an authorization code or access token in the URL. Resource Server URL will be used if empty
• Client Scope - One or more space-separated strings indicating which permissions the application is requesting. The specific OAuth API you’re using will define the scopes that it supports
• Client ID - The public identifier for the application, obtained from your identity provider
• Client Secret - The application’s client secret, obtained from your identity provider. This ensures that the request to get the access token is made only from the application
• Client State - The application generates a random string and includes it in the request. It should then check that the same value is returned after the user authorizes the app. This is used to prevent CSRF attacks
• ExtensionGrant - Can be ignored at this moment
• Username - The username of the end user in case of using the “Resource Owner Password Credentials” or “Client Credentials” grant types
• Password - The password of the end user in case of using the “Resource Owner Password Credentials” or “Client Credentials” grant types
• Username Form - The username for additional form authentication flow on the application
• Password Form - The password for additional form authentication flow on the application
• Never Do Basic Auth - Prevents InsightAppSec from sending HTTP Basic authentication header (base64 encoded Username and Password values) in case of using “Resource Owner Password Credentials” or “Client Credentials”

OAuth Grant Types

Depending on your use case, you will need to use a different OAuth flow. The “grant type” property determines the OAuth flow that your application is using. You can learn more about grant types here: https://oauth.net/2/grant-types/.

The following chart can help you choose the appropriate grant type for your application.

The following sections describe which OAuth properties you need to provide to InsightAppSec based on your grant type.

Additional OAuth Properties

In addition to the properties in the OAuth section of the Authentication screen, there are some Advanced Options that can be helpful to configure the OAuth flow for your application.

• JsonPostBodies - Support JSON format on OAuth Authorization Server requests and responses
• OAuthCustomField - Any additional parameters that should be sent to the Authorization Server

You can authenticate your InsightAppSec scans using the Azure Active Directory Authentication Library (ADAL).

Configure ADAL

To configure ADAL authentication, you must go to Authentication > ADAL screen in the scan config wizard, and set the following options:

• Resource ID - This will depend on your Azure Active Directory (AD) usage. If you are using an Azure Native AD, this should be one of two defaults, either graph.windows.net or graph.microsoft.com. If you are not using a native Azure AD, your AD admin may be able to help you get the correct ID for your instance.
• Tenant - The domain name of your Azure AD tenant, such as example.onmicrosoft.com. This can be found on the Azure Active Directory overview screen.
• Client ID - The Application (client) ID under the app that you have registered.
• Authority URL - Defaults to https://login.microsoftonline.com/ so if left blank this is what will be passed up.
• Username - If you are using the Client ID, Username, and Password authentication flow, this will be the username for your account.
• Password - The password is not the password of your user account, instead it's the secret key that is created under the 'Certificate & secrets' section. You can use ADAL in conjunction with OAuth as your authentication protocol (Reference: https://docs.microsoft.com/en-us/azure/active-directory/develop/v1-oauth2-implicit-grant-flow).

You can authenticate your InsightAppSec scans using the Microsoft Authentication Library (MSAL).

Configure MSAL

To configure MSAL authentication, go to Authentication > MSAL screen in the scan config wizard, and complete the following steps:

1. Click Choose File. Upload the required cert by browsing to the .pfx file. You can also drag and drop.
2. Click Use Selected File.
3. Fill out the following fields:

• X509CertPassword - The password of the cert provided.
• X509CertStoreKey - The file name of the cert provided.
• TenantId - Domain name of tenant.
• ClientId - The Application (client) ID under the app that you have registered.
• Scope - OAuth 2.0 permissions.
• Token Refresh Period (10 minutes minimum) - The frequency at which you want to refresh the token.

The regex is trying to match the text on the page of a 302 redirect from a canary page. If it matches, then the application assumes you are logged out.

Note: The canary page is a URL from your site that can only be reached if you are logged in. You can specify this URL by following the steps here.

A regular expression that matches the text on the link that points to the login page. InsightAppSec will use this regex to find the login link and apply the user-provided credentials. If the user is already logged in, this regex will help InsightAppSec from accidentally crawling the login again.

If the text on your page matches this regular expression, InsightAppSec assumes that you are still logged in. This regex usually matches the sign out link, since the sign-out option is only available if the user is still logged in.

If your application always populates a specific HTTP header when a user is logged-in, you can use a matching regex in this field. If the headers match the regex, InsightAppSec will assume that the user is logged-in.

If the text on your page matches this regular expression, InsightAppSec assumes that the session has expired.

If your application always populates a specific HTTP header when a session expires you can use a matching regex in this field. If the headers match the regex, InsightAppSec assumes the session has been lost/expired.

/details>

You can use the Regex Builder of the Rapid7 AppSec Toolkit to test your regular expressions before using them in InsightAppSec.