Log4Net

Logging To InsightOps

InsightOps provides a plugin for log4net.

To configure your application to log to InsightOps, you will need to perform the following tasks:

• Create a InsightOps account.
• Create a host and log to receive your log data.
• Adding the Plugin library and the appropriate InsightOps Appender libraries to your application.
• Configure the Plugin and the InsightOps appender in your application.
• Send log messages from your application.
• These steps are outlined in further detail below.

Setup

The easiest way to add the log4net and the InsightOps Target libraries to your application is to install the R7Insight.Log4net Nuget package. This package will install the InsightOps Target library and will also automatically install the log4net package as a dependency.

If you would rather install the InsightOps appender manually, you can download the complete code in this GitHub repository, compile the R7InsightLog4net Visual Studio project within it into a DLL file, and then reference this file in your application. If you choose this option you must install log4net yourself.

Configuring log4net and the InsightOps Appender

General log4net configuration is beyond the scope of this readme. Please refer to the Configuration section of the log4net manual for details on how to configure log4net.

log4net allows log messages to be sent to multiple destinations. In log4net terminology, such an output destination is called an appender. Appenders must implement the log4net.Appenders.IAppender interface. The InsightOps Appender library provides such an appender component that is specifically designed to send log messages to InsightOps in an efficient manner.

The InsightOps appender is configured and added to your log4net configuration in the normal way using an element:

 
1
<appender name="InsightAppender" type="log4net.Appender.InsightAppender, R7Insight.Log4net">
2
    ...
3
</appender> 

The InsightOps appender has two categories of settings that are configured somewhat differently:

• Logging settings
• InsightOps credentials

log4net Logging Settings

Logging settings determine how the appender operates, and are specified as child elements of the element. The InsightOps appender supports the following configuration settings:

• Level: The lowest Log4net logging level that should be included. All log messages with a logging level below this level will be filtered out and not sent to InsightOps.
• Debug: Set to true to send internal debug messages to the Log4net internal logger.
• UseSsl: Set to true to use SSL to send data to InsightOps (see below for more information).
• Layout: The layout used to format log messages before they are sent to InsightOps. See the Configuration section of the Log4net manual for more information on configuring layouts. Here is an example of an appender configuration that works well for InsightOps:

 
1
   <appender name="InsightAppender" type="log4net.Appender.InsightAppender, R7Insight.Log4net">
2
        <Debug value="true" />
3
        <Region value="eu" />
4
        <UseSsl value="true" />
5
        <Token value="PLEASE PUT TOKEN HERE" />
6
        <layout type="log4net.Layout.PatternLayout">
7
          <conversionPattern value="%d{ddd MMM dd HH:mm:ss zzz yyyy} %logger %: %level%  - %m"/>
8
        </layout>
9
     </appender>

Next you need to place the following line in your AssemblyInfo.cs file: For web applications use:

 
1
[assembly: log4net.Config.XmlConfigurator(ConfigFile="Web.config", Watch = true)]

For Console apps use:

 
1
[assembly: log4net.Config.XmlConfigurator(ConfigFile="App.config",Watch=true)]
2

log4net InsightOps Credentials

InsightOps credentials determine to which host and log your log messages are sent. The following settings constitute the InsightOps credentials:

Token: The unique token GUID of the log to send messages to. This applies when using the newer token-based logging.

Therefore, the InsightOps credentials can be specified more flexibly than the configuration settings. You have three options:

• Specify the credentials as child elements of the element (if you don’t need the added flexibility).
• Specify the credentials as settings in the element in your App.config och Web.config file.
• Specify the credentials as Windows Azure role configuration settings in your cloud service project (only applicable when running your application as a cloud service in Windows Azure).

The InsightOps appender uses the CloudConfigurationManager class internally to read the credential values. This class looks for each credential value in the following order:

• If the value exists as a Windows Azure role configuration setting, that value is used.
• Otherwise if the value exists as a setting in the element in your App.config och Web.config file, that value is used.
• Otherwise if the value exists as a configured child element of the element, that value is used.
• If the value was not found in any of these locations, errors are logged to the Log4net internal debug log and logging to InsightOps will fail.

Here is an example of how to specify the credentials in the element:

 
1
<appender name="InsightAppender" type="log4net.Appender.R7InsightAppender, R7Insight.Log4net">
2
    <token value="bb61600f-f766-451e-b55f-9204f536a79f" />
3
    ...
4
</appender>

Here is an example of how to specify the credentials in the element in your App.config or Web.config file:

 
1
<appSettings>
2
    <add key="Insight.Token" value="bb61600f-f766-451e-b55f-9204f536a79f" />
3
</appSettings>

Here is an example of how to specify the credentials as Windows Azure role configuration settings:

 
1
<ServiceConfiguration serviceName="MyApp" osFamily="3" osVersion="*" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" schemaVersion="2013-03.2.0">
2
    <Role name="MyRole">
3
        <Instances count="2" />
4
        <ConfigurationSettings>
5
            <Setting name="Insight.Token" value="bb61600f-f766-451e-b55f-9204f536a79f" />
6
        </ConfigurationSettings>
7
    </Role&rt;
8
</ServiceConfiguration>

log4net Logging Context Information in Web Applications

In web application it is often helpful to use Log4net’s built-in ability to log additional contextual information with each log message. This works particularly well in combination with InsightOps’ log message indexer, which can identify any key-value-pairs in the incoming log message and index those for fast search and retrieval.

Here is an example of how additional web-specific contextual log information can be added to the layout of the InsightOps appender in a format that the InsightOps parser will recognize and index:

 
1
<appender name="InsightAppender" type="log4net.Appender. R7InsightAppender, R7Insight.Log4net">
2
    ...
3
    <layout type="log4net.Layout.PatternLayout">
4
        <param name="ConversionPattern" value="%d %logger %level% %m%nSessionId='%aspnet-request{ASP.NET_SessionId}'; Username='%aspnet-request{AUTH_USER}'; ClientIpAddress='%aspnet-request{REMOTE_ADDR}'; ClientUserAgent='%aspnet-request{HTTP_USER_AGENT}'; ServerName='%aspnet-request{SERVER_NAME}'; RequestMethod='%aspnet-request{REQUEST_METHOD}'; RequestUrl='%aspnet-request{URL}'; RequestQueryString='%aspnet-request{QUERY_STRING}'; RequestCookies='%aspnet-request{HTTP_COOKIE}';%n" />
5
    </layout>
6
</appender>

Token-Based Logging with log4net

Our recommended method of sending messages to InsightOps is via Token TCP over port 10000. To use this method, select Token TCP as the source type when creating a new log in the InsightOps UI, and then paste the token that is printed beside the log in the value for the Insight.Token credential setting.

Sending Log Data over SSL/TLS with log4net

The InsightOps appender supports sending log data over SSL/TLS with both of the above logging methods by setting the useSsl logging setting to true in the appender definition. This is more secure but may have a performance impact.

Sending Log Messages from Your Application with log4net

With installation and configuration out of the way, you are ready to send log data to InsightOps.

In each class you wish to log from, add the following using directive at the top if it’s not already there:

 
1
using log4net;

Then create a logger object at the class level:

 
1
private static readonly ILog m_Logger = LogManager.GetLogger(typeof({YOURCLASSNAMEHERE}).FullName);

Be sure to enter the name of current class instead of above. This creates a logger with the same name as the current class, which organizes the log4net configuration hierarchy according to your code namespace hierarchy. This provides both clarity when reading the logs, and convenience when configuring different log levels for different areas of your code.

Now within your code in that class, you can log using log4net as normal and it will log to Rapid7 Insight.

Examples:

 
1
m_Logger.Debug("Debugging message");
2
m_Logger.Info("Informational message");
3
m_Logger.Warn("Warning message");
4
m_Logger.Error("Error message", ex);

Complete code example:

 
1
using log4net;
2

3
public class HomeController : Controller
4
{
5
    private static readonly ILog m_Logger = log4net.LogManager.GetLogger(typeof(HomeController).FullName);
6

7
    public ActionResult Index()
8
    {
9
        m_Logger.Debug("Home page viewed.");       
10
        ViewBag.Message = "Welcome to ASP.NET MVC!";       
11
        m_Logger.Warn("This is a warning message!");       
12
        return View();
13
    }
14
}

Troubleshooting log4net

By default the InsightOps appender logs its own debug messages to log4net’s internal logger. Checking these debug messages can help figuring out why logging to InsightOps does not work as expected.

To disable log4net internal debug messages, set the log4net.Internal.Debug setting in the section of your App.config or Web.config file to false:

 
1
<appSettings>
2
    <add key="log4net.Internal.Debug" value="false" />
3
</appSettings>

If you would like to keep log4net internal debugging enabled in general, but disable Insight logger debug messages specifically, then change the debug parameter inside the element to false instead:

 
1
<appender name="LeAppender" type="log4net.Appender.InsightAppender, R7Insight.Log4net">
2
    <debug value="false" />
3
    ...
4
</appender>

Ensure that you followed the section of this readme regarding your AssemblyInfo.cs file.

Shutting Down log4net’s Logger

The InsightOps appender keeps an internal queue of log messages and communicates with the InsightOps system using a background thread which continuously sends messages from this queue. Because of this, when an application is shutting down, it is possible that some log messages might still remain in the queue and will not have time to be sent to InsightOps before the application domain is shut down.

To work around this potential problem, consider adding the following code to your application, which will block for a moment to allow the InsightOps appender to finish logging all messages in the queue. The AreAllQueuesEmpty() blocks for a specified time and then returns true or false depending on whether the queues had time to become empty before the method returns.

public
1
{
2
    // This will give the background thread some time to finish sending messages to InsightOps.
3
    var numWaits = 3;
4
    while (!R7InsightCore.Net.AsyncLogger.AreAllQueuesEmpty(TimeSpan.FromSeconds(5)) && numWaits > 0)
5
        numWaits--;
6
}