Authentication

Web applications often have a section for registered users only. This part of the web application can only be assessed by logging in to the app, and the process for logging in is called "Authentication". AppSpider supports a number of ways to authenticate into your application. You can configure these authentication methods in the Authentication tab of the scan config wizard.

Simple Form Authentication

As the name suggests, this technique helps you sign in to websites that use simple form-based authentication. These websites have a login form that consists of a username field, a password field, and a submit button. You can refer to this wikipedia article to learn more about form based authentication: https://en.wikipedia.org/wiki/HTTP%2BHTML_form-based_authentication.

If your web application uses simple form authentication, you can configure the authentication settings using the following steps:

  1. Open the Authentication > Simple Form Authentication tab.
  2. Select the Site requires Form authentication checkbox.
  3. Enter your username and password in the respective fields. Re-type your password in the "Confirm Password" field.
  4. If you use a single sign-on solution like Okta, and the login page exists outside of the domain restrictions set up in the scan, enter the URL of the login page in the "SSO" field.

Macro Authentication

AppSpider may sometimes be unable to reach the login page of your application, or the login form may become available only after a certain specific sequence of actions has been carried out on your website. For example, the login form may be in a pop-up that gets dynamically generated with Javascript when the "Login" button is pressed from the "Administration" window. You can enable AppSpider to perform this sequence of steps by recording a macro.

A macro is a sequence of actions such as the clicking of buttons or text entry in a web page. AppSpider records these sequences in xml format within a .rec file. During a scan, AppSpider can replay the actions in this file to log in to the web application. If you wish to use macro authentication, you can configure it using the following steps:

  1. Open the Authentication > Macro Authentication tab.
  2. Select the Use login macro (for Form Authentication) checkbox.
  3. If you have previously recorded a login macro on this system, you can select it by opening the file explorer with the ellipsis (...) button. If you do not have a previously recorded macro, follow the next steps.
  4. Click the Record button. This will open the Browser Macro Recorder in a new tab. The Browser Macro Recorder will open your target application by default.
  1. Follow the steps for logging in to your application. Save the macro into a .rec file on your local system. By default, AppSpider will save the macro file in the "Macro" folder under the AppSpider data directory.

Note

As AppSpider records your actions and translates them into XPath commands, do not perform any actions apart from those required for logging in while recording the login macro. Additionally we advise that you click into each field you are injecting data into instead of using the “Tab” or “Return” buttons. During a scan, the engine will replay the macro and then look for the patten from the "Advanced Options > Logged-in Regex" field in the page on your screen. Once AppSpider confirms that the authentication was successful, it will move forward with the scan.

If the indicator for a logged-in state is hidden under a menu item, expand it so that AppSpider can examine it and check the logged-in state. This is especially important if your application is built using modern frameworks such as Angular, React or Ember. Often times these frameworks will render these fields using javascript which can not be seen within the Document Object Model (DOM). If you are having issues with getting the “Logged-in Regex” to work we recommend that you enable traffic logs and inspect the html response returned at the end of the macro. This will allow you to see what the scanner is seeing and help you create a regex that will work.

  1. Close the Browser Macro Recorder tool and return to the Scan Config wizard.
  2. Optionally, you can test the macro by replaying it using the Test button.
  3. If your macro does not work correctly and you wish to debug it, you can select the Display Macro Replay checkbox. During the scan, a browser window will open and show you all the steps carried out by AppSpider while replaying your macro. This will let you figure out where the engine runs into trouble and you can fix your macro recording accordingly.
  4. Determine whether the Chrome or Internet Explorer browser displays your website correctly, and select the respective entry from the Browser dropdown. The default browser is Chrome.

Note

The Browser setting configures the browser used for playback of the macro. If you wish to change the browser used for recording the macro, click the Options menu of the Menu Bar and go to Options > Environment > Macro Recording > Browser Engine.

HTTP Authentication

The HTTP protocol supports authentication using a username and password. You can use this reference article to learn more about HTTP Authentication: https://docs.microsoft.com/en-us/dotnet/framework/wcf/feature-details/understanding-http-authentication

AppSpider supports the Basic, NTLM, and Kerberos protocols for HTTP authentication. If your target app uses HTTP authentication, you can configure the authentication process using the following steps:

  1. Open the Authentication > HTTP Authentication tab.
  2. Select the Site requires HTTP authentication (Basic, NTLM, or Kerberos) checkbox to enable HTTP authentication.
  3. Enter your Username and Password in the respective fields. If the username and password are the same as those entered in the Simple Form Authentication section, enable the Use Form credentials checkbox.

Note

NTLM authentication will require the user account's domain and username, both of which are case sensitive. Forgetting to include the domain or using the incorrect case will cause the authentication action to fail

Scan Bootstrap

Many modern web applications use security measures like two-factor authentication that require manual intervention for logging in to the application. For example, suppose your application requires you to enter a one time password sent to you via SMS, or the text in a CAPTCHA box. You can use Scan Bootstrap (common referred to as Bootstrap) authentication in these cases.

During the scan, a browser window will open and allow you to manually guide the scanner to the login area and complete all the steps to log the scanner into the web site. After the login process has been completed, AppSpider can continue scanning your application for vulnerabilities in an automated fashion. If the scan engine gets logged out during the scan, it will trigger bootstrap authentication to re-authenticate in to the application. In this case, a browser window will pop up again so you can manually log in to the application. You can use the following steps to configure Bootstrap authentication:

  1. Open the Authentication > Scan Bootstrap tab.
  2. Select the Allow Scan Bootstrap checkbox.
  3. Set the amount of time in seconds that you feel you will require for logging in to your application. The default value is 60 seconds.

Note

During the scan, carry out only the steps required for logging into the application. After the login process is completed, leave the browser window open as AppSpider will close it automatically. AppSpider will look for the patten from the "Advanced Options > Logged-in Regex" field in the web page open in the browser. If the indicator for logged-in state is hidden under a menu item, expand it so that AppSpider can examine it and check the logged-in state. Once it confirms that the authentication was successful, it will move forward with the scan.

Selenium Script Authentication

Selenium is a framework for automated testing of web applications. Users can record actions like entering data in forms and clicking buttons using Selenium and replay them on demand to ensure that the web application behaves as desired.

AppSpider supports authentication using Selenium files, so you can record the actions needed to log in to your application in a Selenium script. During the scan, AppSpider can replay the actions in this script to log in to the web application.

Use the following steps to authenticate with a Selenium script:

  1. Open the Authentication > Selenium Script Authentication tab.
  2. Select the Use Selenium script (for Form Authentication) checkbox.
  3. Select the Web Driver (reference: https://www.seleniumhq.org/docs/03_webdriver.jsp) for your Selenium script. AppSpider is installed with the Chrome webdriver which will be used by default.
  4. Click the ellipsis (...) button next to the Selenium Script File text box. This opens the file explorer. Navigate to a Selenium file of a supported format and select it.
  5. Optionally test the Selenium script authentication by replaying it using the Test button. If authentication is successful, you will see a popup that says "Selenium Login executed successfully".

The following are some examples of Selenium authentication scripts in different formats:

Note

You must use port 32768 for Selenium authentication scripts.

HTML - Selenium Script

xml
1
<?xml version="1.0" encoding="UTF-8"?>
2
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
3
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
4
<head profile="http://selenium-ide.openqa.org/profiles/test-case">
5
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
6
<link rel="selenium.base" href="http://webscantest.com/" />
7
<title>selenium</title>
8
</head>
9
<body>
10
<table cellpadding="1" cellspacing="1" border="1">
11
<thead>
12
<tr><td rowspan="1" colspan="3">selenium</td></tr>
13
</thead><tbody>
14
<tr>
15
<td>open</td>
16
<td>/</td>
17
<td></td>
18
</tr>
19
<tr>
20
<td>clickAndWait</td>
21
<td>link=Login</td>
22
<td></td>
23
</tr>
24
<tr>
25
<td>type</td>
26
<td>name=login</td>
27
<td>admin</td>
28
</tr>
29
<tr>
30
<td>type</td>
31
<td>name=passwd</td>
32
<td>admin</td>
33
</tr>
34
<tr>
35
<td>clickAndWait</td>
36
<td>name=submit_login</td>
37
<td></td>
38
</tr>
39
</tbody></table>
40
</body>
41
</html>
42
43

Jar - Java Selenium Script

java
1
package automationFramework;
2
3
import java.util.concurrent.TimeUnit;
4
import org.openqa.selenium.By;
5
import org.openqa.selenium.Keys;
6
import org.openqa.selenium.Proxy;
7
import org.openqa.selenium.WebDriver;
8
import org.openqa.selenium.chrome.ChromeDriver;
9
import org.openqa.selenium.chrome.ChromeOptions;
10
11
public class TestCase {
12
public static void main(String[] args) throws InterruptedException {
13
ChromeOptions options = new ChromeOptions();
14
Proxy proxy = new Proxy();
15
proxy.setHttpProxy("127.0.0.1:32768");
16
proxy.setSslProxy("127.0.0.1:32768");
17
options.setProxy(proxy);
18
WebDriver driver = new ChromeDriver(options);
19
driver.manage().timeouts().implicitlyWait(60,TimeUnit.SECONDS);
20
driver.navigate().to("http://webscantest.com/login.php");
21
driver.findElement(By.cssSelector("input[name=login]")).sendKeys("admin");
22
driver.findElement(By.cssSelector("input[name=passwd]")).sendKeys("admin");
23
driver.findElement(By.cssSelector("input[name=passwd]")).sendKeys(Keys.ENTER);
24
Thread.sleep(3000);
25
driver.quit();
26
}
27
}

Exe - C# Selenium Script

csharp
1
using System;
2
using OpenQA.Selenium;
3
using OpenQA.Selenium.Support.UI;
4
using OpenQA.Selenium.Chrome;
5
//username test_user
6
//password 123456
7
namespace Selenium_Demo
8
{
9
class Program
10
{
11
static void Main(string[] args)
12
{
13
string URL = "http://hackazon.webscantest.com/user/login";
14
ChromeOptions options = new ChromeOptions();
15
ChromeOptions headlessOptions = new ChromeOptions();
16
string appSpider_port = "32768";
17
Proxy proxy = new Proxy
18
{
19
HttpProxy = "127.0.0.1:" + appSpider_port,
20
SslProxy = "127.0.0.1:" + appSpider_port
21
};
22
options.Proxy = proxy;
23
headlessOptions.Proxy = proxy;
24
//THIS CHECKS IF THE LOGIN SCRIPT IS RUNNING IN APPSPIDER OR NOT (CHROME ONLY)
25
headlessOptions.AddArguments("headless");
26
IWebDriver driver = new ChromeDriver(headlessOptions);
27
driver.Url = URL;
28
if (driver.PageSource.Equals("<html xmlns=\"http://www.w3.org/1999/xhtml\"><head></head><body></body></html>"))
29
{
30
options.Proxy = null;
31
}
32
driver.Quit();
33
//THIS CHECKS IF THE LOGIN SCRIPT IS RUNNING IN APPSPIDER OR NOT (CHROME ONLY)
34
driver = new ChromeDriver(options);
35
driver.Url = URL;
36
driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(60);
37
////GATHER YOUR ELEMENTS
38
IWebElement userName = driver.FindElement(By.Id("username"));
39
IWebElement password = driver.FindElement(By.CssSelector("input[placeholder='Password']"));
40
////GATHER YOUR ELEMENTS
41
////EXECUTE THE LOGIN
42
userName.SendKeys("test_user");
43
password.SendKeys("123456");
44
password.SendKeys(Keys.Enter);
45
////EXECUTE THE LOGIN
46
////WAIT FOR AN ELEMENT THAT APPEARS WHEN YOU'RE LOGGED IN
47
IWebElement loggedInElement = driver.FindElement(By.LinkText("Logout"));
48
////WAIT FOR AN ELEMENT THAT APPEARS WHEN YOU'RE LOGGED IN
49
driver.Quit();
50
}
51
}

Proxy Log Authentication

You can authenticate into your applications using a web proxy tool such as the Traffic Recorder. Using the proxy tool, you can record the interactions (e.g. HTTP GET and POST requests) between the front end application and the back end server in a proxy file. AppSpider can replay these interactions to authenticate into your application. Proxy log files can be of the following formats:

  • AppSec toolkit Traffic Files (*.trec)
  • Burp Files (*.xml)
  • Paros Files (*.txt)
  • WebScarab Files (conversationlog)
  • HAR (HTTP Archive) Files (*.har)
  • Fiddler Files (*.saz)

Use the following steps to authenticate with a Traffic file:

  1. Open the Authentication > Macro Authentication tab.
  2. Select the Use Proxy Log (for Form Authentication) checkbox.
  3. If you do not have a proxy file already, use the Traffic Recorder tool to record the interactions involved in the login process.
  4. Click the ellipsis (...) button beside the Proxy Log File textbox. This will open the file explorer. Navigate to a proxy file of a supported format and select it.
  5. Optionally test the proxy log authentication by replaying it using the Test button. If authentication is successful you will see a popup that says "Proxy Login executed successfully".

Session Hijacking

If your web application uses session cookies for maintaining a logged-in state, you can capture this cookie and use the Session Hijacking method for authentication. Use the following steps to set up this authentication method:

  1. Capture the session cookies related to authentication using your browser’s developer tool.
  2. Open the Authentication > Session Hijacking (relogin not supported) tab.
  3. Select the Session Hijacking checkbox.
  4. Click the Utilize Captured Session Cookie link. The “Session settings” window will appear.
  5. Enter the session cookie from step 1 in the Session Cookie field. Separate multiple cookies with a semicolon. For example, consider the following cookies captured from http://webscantest.com:
  1. If these cookie values need to be maintained throughout the scan, you can select the Lock cookie values for duration of scan checkbox. You can apply this setting to all cookies using the Apply to all cookies checkbox, or enter specific cookies in the table.
  2. Click Ok to return to the “Authentication” screen.

This is the least preferred authentication method, since the scanner cannot re-authenticate in case it loses the session during the scan.

OAUTH

OAuth (https://oauth.net/) is an authorization method that is used by applications to grant fine grained access to clients. AppSpider supports OAuth 2.0 which is the industry-standard protocol for authorization. OAuth 2.0 focuses on client developer simplicity while providing specific authorization flows for web applications, desktop applications, mobile phones, and living room devices. If your application has granted AppSpider the access to certain capabilities, you can enter the required details in the Authentication > OAUTH tab. When starting a scan, AppSpider can provide these details to your application and receive an access token.

Depending on your use case, you will need to use a different OAuth flow. The “grant type” property determines the OAuth flow that your application is using. You can learn more about grant types here: https://oauth.net/2/grant-types/.

AppSpider supports the following grant types:

  1. Authorization Code
  2. Implicit
  3. Resource Owner Password Credentials
  4. Client Credentials

The following chart can help you choose the appropriate grant type for your application.

Set Up OAuth Authentication

If you want to scan an application that uses OAuth, you will need to know the grant type used by your application. You can usually get this information from your application developers. If you examine the traffic from a connection, you can also often see the grant type in the URL.

The Authentication > OAUTH screen has a number of options to configure your application's OAuth properties. You will need to select the OAuth enabled option to enable OAuth authentication, and then provide the required values based on your grant type.

  • Resource Server URL - The identifier for your API server. The resource server handles authenticated requests after the application has obtained an access token
  • Authorization Server URL - The authorization server URL, obtained from your identity provider (reference: https://en.wikipedia.org/wiki/Identity_provider)
  • Redirect URI - The URL that the authorization server will redirect the user back to, with an authorization code or access token in the URL. Resource Server URL will be used if empty
  • Client Scope - One or more space-separated strings indicating which permissions the application is requesting. The specific OAuth API you’re using will define the scopes that it supports
  • Client Id - The public identifier for the application, obtained from your identity provider
  • Client Secret - The application’s client secret, obtained from your identity provider. This ensures that the request to get the access token is made only from the application
  • Client State - The application generates a random string and includes it in the request. It should then check that the same value is returned after the user authorizes the app. This is used to prevent CSRF attacks
  • Username - The username of the end user in case of using the “Resource Owner Password Credentials” or “Client Credentials” grant types
  • Password - The password of the end user in case of using the “Resource Owner Password Credentials” or “Client Credentials” grant types

OAuth Grant Types

The following sections describe which OAuth properties you need to provide to AppSpider based on your grant type.

Authorization Code

The Authorization Code grant type is used by web and mobile apps. It differs from most of the other grant types by first requiring the app launch a browser to begin the flow. At a high level, the flow has the following steps:

  • The application opens a browser to send the user to the OAuth server
  • The user sees the authorization prompt and approves the app’s request
  • The user is redirected back to the application with an authorization code in the query string
  • The application exchanges the authorization code for an access token

The Authorization Code flow is best used by server-side apps where the source code is not publicly exposed. The apps should be server-side because the request that exchanges the authorization code for a token requires a client secret, which will have to be stored in your client. The server-side app requires an end-user, however, because it relies on interaction with the end-user’s web browser which will redirect the user and then receive the authorization code. See https://developer.okta.com/authentication-guide/auth-overview/#authorization-code-flow and https://developer.okta.com/authentication-guide/implementing-authentication/auth-code for more information.

Mandatory properties

  • Resource Server URL
  • Client Id

Implicit

The Implicit grant type is a simplified flow that can be used by public clients, where the access token is returned immediately without an extra authorization code exchange step.

It is generally not recommended to use the implicit flow (and some servers prohibit this flow entirely). In the time since the spec was originally written, the industry best practice has changed to recommend that public clients should use the authorization code flow with the PKCE extension instead.

Mandatory properties

  • Resource Server URL
  • Client Id

NOTE: PKCE is not supported yet

Resource Owner Password Credentials

The resource owner password credentials grant type is suitable in cases where the resource owner has a trust relationship with the client, such as the device operating system or a highly privileged application. The authorization server should take special care when enabling this grant type and only allow it when other flows are not viable. See https://tools.ietf.org/html/rfc6749#4.3 for more information.

Mandatory properties

  • Resource Server URL
  • Username
  • Password

Client Credentials

The Client Credentials grant is used when applications request an access token to access their own resources, not on behalf of a user. The client needs to authenticate themselves for this request. Typically the service will allow either additional request parameters Client Id and Client Secret, or accept the Client Id and Client Secret in the HTTP Basic auth header.

Mandatory properties

  • Resource Server URL

Additional OAuth Properties

In addition to the properties in the OAuth section of the Authentication screen, there are some Advanced Options that can be helpful to configure the OAuth flow for your application.

  • Resource Owner URL - An entity capable of granting access to a protected resource. Optional, in most cases should be equal to Resource Server URL
  • UsernameForm/PasswordForm - The username and password for additional form authentication flow on the application
  • ExtensionGrant - Can be ignored at this moment
  • AuthorizationGrantType - Digital representation of the supported grant types
  • NeverDoBasicAuth - Prevents AppSpider from sending HTTP Basic authentication header (base65 encoded Username and Password values) in case of using “Resource Owner Password Credentials” or “Client Credentials”. Enabled by default (value is 0)
  • JsonPostBodies - To support JSON format on OAuth Authorization Server requests and responses
  • Azure* - All properties with Azure at the beginning are obsolete and should not be used
  • OAuthCustomField - To support any additional parameters that should be sent to the Authorization Server

Note

In most cases, except the Implicit grant type, OAuth authentication can be bypassed using a macro. If you are having trouble configuring OAuth with AppSpider, you should try Macro authentication.

Advanced Settings

While attempting authenticated scanning of an app, AppSpider needs a way to learn that authentication has been successful. It attempts to deduce the logged-in state of the app by examining the headers and body of web pages. The fields in the Authentication > Advanced tab can be used to train AppSpider to recognize the logged-in state of your application. You can use the Regex Builder to test your regular expressions before using them in AppSpider.

  • Configure SSL certificate link - Opens SSL configuration window where user may configure the SSL certificate
  • Logged-in Regex - If the text on your page matches this regular expression, AppSpider assumes that you are still logged in. This regex usually matches the "Sign out" link, since the sign-out option is only available if the user is still logged in.
  • Assume Good Login - You may sometimes be unable to find a regex that matches the logged-in state of the app. There may be multiple login links leading to different areas of the product, and AppSpider might be attempting the same credentials everywhere leading to account lockouts. You can enable the Assume Good Login checkbox to instruct AppSpider not to check for logged-in state after the initial log in.