NXLog

Overview

NXLog is an open source tool that can convert log data into JSON for easy searching and analysis.

NXLog can be configured to write to a new log file, or to send data directly to InsightOps.

Installation & Configuration

Download and install the latest version of nxlog, which you can find here. Please install nxlog locally and set the ROOT to the folder in which your nxlog was installed, otherwise nxlog will not start.

Below is a sample configuration file. Please see the nxlog reference manual about additional configuration options.

Once installed, open the Nxlog configuration file located at: C:\Program Files (x86)\nxlog\conf\nxlog.conf and paste the following into to the file, adjusting for your account as necessary:

text
1
## This is a sample configuration file. See the nxlog reference manual about the
2
## configuration options. It should be installed locally and is also available
3
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
4
5
## Please set the ROOT to the folder your nxlog was installed into,
6
## otherwise it will not start.
7
8
#define ROOT C:\Program Files\nxlog
9
define ROOT C:\Program Files (x86)\nxlog
10
11
Moduledir %ROOT%\modules
12
CacheDir %ROOT%\data
13
Pidfile %ROOT%\data\nxlog.pid
14
SpoolDir %ROOT%\data
15
LogFile %ROOT%\data\nxlog.log
16
17
# Include fileop when rotating logs or while debugging, also enable in the output module below
18
#<Extension fileop>
19
# Module xm_fileop
20
#</Extension>
21
22
# Create the parse rule for IIS logs. You can copy these from the header of the IIS log file.
23
<Extension w3c>
24
Module xm_csv
25
Fields $date, $time, $s_ip, $cs_method, $cs_uri_stem, $cs_uri_query, $s_port, $cs_username, $c_ip, $cs_User_Agent, $cs_Referer, $sc_status, $sc_substatus, $sc_win32_status, $time_taken
26
FieldTypes string, string, string, string, string, string, integer, string, string, string, string, integer, integer, integer, integer
27
Delimiter ' '
28
</Extension>
29
30
<Extension json>
31
Module xm_json
32
</Extension>
33
34
<Extension syslog>
35
Module xm_syslog
36
</Extension>
37
38
<Input internal>
39
Module im_internal
40
Exec $Message = to_json();
41
</Input>
42
43
<Input eventlog>
44
#This is the Windows Event Log Section - for 2008 and above use im_msvistalog - for 2003 and earlier, use im_mseventlog
45
#
46
Module im_msvistalog
47
# For windows 2003 and earlier use the following:
48
# Module im_mseventlog
49
50
# Prepend the JSON event with the log token if you're sending directly to InsightOps
51
Exec $raw_event = "<LOG TOKEN GOES HERE>" + to_json();
52
53
# If you're writing to a log file, then no need for the token
54
# Exec $raw_event = to_json();
55
56
</Input>
57
58
<Output eventlog_out>
59
# use this module to write to a text file that the agent can send in
60
#Module om_file
61
#file 'c:\test\eventlog.txt'
62
#Rotate created files
63
#<Schedule>
64
#Every 1 hour
65
#Exec file_cycle('c:\test\eventlog.txt', 2);
66
#Exec eventlog_out->reopen();
67
#</Schedule>
68
69
# send log entries directly to InsightOps
70
Module om_tcp
71
Host ENDPOINT
72
Port PORT
73
</Output>
74
75
76
<Route EventLog>
77
Path eventlog => eventlog_out
78
</Route>
79

Set up Event Source

Log in to InsightOps Click the “Add Data” link in top navigation Click “Quick add” Create a new log using Token TCP option -Make note of the token and endpoint that are displayed when the log is created

Configure NXLog

Replace the nxlog.conf file with the sample above. Replace the “” string with the log token you noted earlier Replace ENDPOINT with the endpoint from the UI i.e. REGION.data.logs.insight.rapid7.com where region is your region (eg. eu,us, ca etc) Replace PORT with 10000 for Token TCP. Restart NXLog service in the windows services control panel.

Restart the Nxlog service

Open the services tool in the start menu. Search for nxlog in the services and then select restart. This will restart nxlog and follow the new configuration.

Troubleshooting

If you find that the nxlog is not sending data, information can be found in the nxlog at File C:\\Program Files (x86)\\nxlog\\data\\nxlog.log