IIS Reverse Proxy Installation

Background

While deploying IIS and IIS agent with locally hosted applications is a popular option, often times one wants to configure IIS as a reverse proxy. This guide describes how to install and configure IIS agent in a IIS reverse proxy deployment end-to-end.

Prerequisites

1. Internet Information Services (IIS). For simplicity, this guide provides steps using IIS 10.0, but the steps should largely work for other versions 7.0+ with minor modifications.
2. Microsoft Application Request Routing (ARR) 3 module. This module allows IIS to provide reliable and scalable routing to backend applications, and is one of the standard modules used when configuring IIS in a reverse proxy format.
3. IIS URL Rewrite Module 2.
4. cURL. This popular tool is used to verify the configuration.

Terminology

For consistency, this document uses the same terminology as Microsoft's Reverse Proxy deployment guides.

Scope

For simplicity, this document describes deployment of a single internal web application which is monitored and managed as a single tCell Application ID. In addition, to make initial development setup faster and easier to test, the examples below assume a simple plain http configuration. It is strongly recommended before enabling the reverse proxy server for production that https is enabled and http is either disabled or redirected to the https port.

Installation

Configure ARR Server Farm

ARR is IIS's standard way to configure routing from IIS to another server, in this case an internal web application. Follow the steps as outlined in Microsoft's ARR Server Farm Guide.

Configure and Download tCell Agent Artifacts

1. Create a tCell Application
2. Make sure output IP connections are supported
3. Make sure you have an Agent API Key.
4. Download agent and config
   • Click on "Admin" in tCell console top navigation bar
   • Click "Download Agent"
   • Select the "Web Server Agent" tab
   • Select "IIS"
   • Download the msi installer
   • Click "Download Config File" to download the agent's config file, using the Application ID and API Key from steps 1 and 3 above.

Install the tCell Agent

To manually install the agent:

1. Run the msi installer and follow the instructions. This will install the agent and put the required files in the directory you specify. By default, the directory is C:\Program Files\Rapid7, Inc\tCell IIS Agent.
2. Place the "tcell_agent.config" file in the root directory of your default web site. In most cases, this will be "C:\inetpub\wwwroot".

Configure the TCELL_AGENT_HOME directory

The tCell agent maintains a local agent cache and writes to log files, which requires write permissions. For security reasons, following the principle of least privilege, IIS default web site file system does not grant write permissions to most non-system administrators.

To both enable write access to important cache and log directories and maintain the principle of least privilege, it is recommended that you configure the TCELL_AGENT_HOME environment variable which will cause the cache and log files to be written to that directory.

1. Create a directory, c:\tcellhome
2. Set the environment variable TCELL_AGENT_HOME to c:\tcellhome

Configure URL Rewrite Module

Follow the instructions in section "Creating URL rewrite rules for your farm" at https://techcommunity.microsoft.com/t5/iis-support-blog/application-request-routing-and-url-rewrite-part-1-server-farms/ba-p/347924 to configure, at server level, to route HTTP requests from default web site to the ARR server farm configured above.

Restart IIS and Windows

In Internet Information Services manager click the Restart link in upper right window. Also restart the Windows operating system to make sure the environment variables configured in previous steps take effect.

Send test requests

To fully initialize the tCell agent, one must have it process HTTP requests. This can be done simply via curl in a command prompt as:

curl -v http://127.0.0.1/

Note 1: Many web servers and API servers check and validate the HTTP Host header, to make sure the incoming request was targeted to the given server. In such cases, the example above will not work as the Host header will contain the proxy server IP address, not the actual web server's hostname. In such circumstances one can use curl's -H flag to manually set the Host header. For example, if the application server has hostname www.foo.com, then one would use:

curl -v -H "Host: www.foo.com" http://127.0.0.1/

Note 2: The above examples all implicitly are based on a backend web application that serves HTTP GET requests on the root, "/", path. For other backend scenarios which either only support other HTTP methods or non-root paths, follow the cURL REST guide to use the appropriate flags and paths.

After running this request, verify the agent is processing ok and sending data to the tCell cloud as follows:

1. Check there is a "tcell.log" file in c:\tcellhome that has a line like Tcell::Processing::RequestDataProcessing::Run:42 INFO Thread started
2. Log into the tCell Console and choose the application created above. Within this, choose the Agent Dashboard and you should see an IIS agent online. Note: due to the highly asynchronous nature of tCell platform, it can take up to 5 minutes for the agent to show online in the console.

Prevent IIS recycling on IIS 7.5 or later

If you’re running IIS 7.5 or later, IIS will recycle your application if it's idle for 20 minutes or longer. When this happens, the agent will unload, stop running, and will show as offline in the tCell console. If a request comes in, the agent will start again. If you don't want IIS to recycle the application when it's idle, you can change your application’s start mode and idle-timeout for your application pool. Note that IIS will still recycle based on its regular recycle schedule. By default, this is set to 1740 minutes, or 29 hours.

To change your Start Mode and Idle Time-out:

1. Go to advanced settings for your application pool by right clicking on you application pool and selecting advanced settings
2. Set the Start Mode to AlwaysRunning, which is found under “General”
3. Set the Idle Time-out (in minutes) to 0. This can be found under Process Model.

Troubleshooting

Failed Request Tracing

Often times, when testing an initial IIS reverse proxy deployment, the system does not behave as expected. For example, when one expects a GET request to succeed and return HTTP 2xx, instead IIS returns an unexpected error result and it is not obvious how to troubleshoot.

IIS's Failed Request Tracing is a very helpful tool for tracing and debugging complex IIS reverse proxy use cases. Microsoft has several useful guides that provide strategies to troubleshoot the end-to-end behavior based on the Failed Request Tracing logs.

Logging

There are three logging types used during different actions in tCell:

• tCell agent logging is used when the agent is running. These are the logs you can set the path for.
• IIS Agent debug logging is used when there are unexpected failures while loading the tCell agent. These logs are stored in C:\tcell.
• Startup error logging is used when the agent can be started, but fails to start. This logging type is disabled by default.

tCell agent log

As of IIS agent v2.0.0, the default location of the log and cache files is C:\ProgramData\Rapid7, inc\tCell IIS Agent\[IIS application id].

Log level can be customized in tcell_agent.config file by adding a block such as the following:

json
1
"logging_options": {
2
    "enabled": true,
3
    "level": "DEBUG"
4
}

IIS Agent debug log

If no logs appear in the log directory you can create the directory C:\tcell\. This is the default directory for IIS agent debug logs.

1. Add the user running the IIS application as full control to the directory.
2. Restart IIS
3. A log file should be created with information about the agent startup.

Startup errors

To enable startup error logging set the TCELL_AGENT_STARTUP_ERROR_LOGGING environment variable to true or specify a valid path. This flag will create a file called tcell_critical_error.log in the root folder of the application running or at the path specified if there's an error at startup.