NXLog

NXLog is an open source tool that can convert log data into JSON for easy searching and analysis.

NXLog can be configured to write to a new log file, or to send data directly to InsightOps.

Installation and configuration

Download and install the latest version of NXLog. Follow the steps from the NXLog documentation here: (http://nxlog.co/products/nxlog-community-edition/download)

Installation folder requirements

Make sure to install NXLog locally and set the ROOT to the folder in which it was installed. NXLog will not start if this installation requirement is not met.

Please see the NXLog reference manual about additional configuration options: http://nxlog.org/docs/

Once installed, open the NXLog configuration file located at: C:\Program Files (x86)\nxlog\conf\nxlog.conf and paste this sample text into the file, adjusting the values for your account as necessary:

Sample configuration file:

text
1
## This is a sample configuration file. See the nxlog reference manual about the
2
## configuration options. It should be installed locally and is also available
3
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
4


5
## Please set the ROOT to the folder your nxlog was installed into,
6
## otherwise it will not start.
7


8
#define ROOT C:\Program Files\nxlog
9
define ROOT C:\Program Files (x86)\nxlog
10


11
Moduledir %ROOT%\modules
12
CacheDir %ROOT%\data
13
Pidfile %ROOT%\data\nxlog.pid
14
SpoolDir %ROOT%\data
15
LogFile %ROOT%\data\nxlog.log
16


17
# Include fileop when rotating logs or while debugging, also enable in the output module below
18
#<Extension fileop>
19
# Module xm_fileop
20
#</Extension>
21


22
# Create the parse rule for IIS logs. You can copy these from the header of the IIS log file.
23
<Extension w3c>
24
    Module xm_csv
25
    Fields $date, $time, $s_ip, $cs_method, $cs_uri_stem, $cs_uri_query, $s_port, $cs_username, $c_ip, $cs_User_Agent, $cs_Referer, $sc_status, $sc_substatus, $sc_win32_status, $time_taken
26
    FieldTypes string, string, string, string, string, string, integer, string, string, string, string, integer, integer, integer, integer
27
    Delimiter ' '
28
</Extension>
29


30
<Extension json>
31
 Module xm_json
32
</Extension>
33
 
34
<Extension syslog>
35
 Module xm_syslog
36
</Extension>
37
 
38
<Input internal>
39
 Module im_internal
40
 Exec $Message = to_json(); 
41
</Input>
42


43
<Input eventlog>
44
#This is the Windows Event Log Section - for 2008 and above use im_msvistalog - for 2003 and earlier, use im_mseventlog
45
#
46
    Module      im_msvistalog
47
# For windows 2003 and earlier use the following:
48
#   Module      im_mseventlog
49


50
# Prepend the JSON event with the log token if you're sending directly to InsightOps
51
Exec $raw_event = "<LOG TOKEN GOES HERE>" + to_json();
52


53
# If you're writing to a log file, then no need for the token
54
# Exec $raw_event = to_json();
55
 
56
</Input>
57


58
<Output eventlog_out>
59
# use this module to write to a text file that the agent can send in
60
    #Module      om_file
61
	#file		'c:\test\eventlog.txt'
62
	#Rotate created files
63
	#<Schedule>
64
		#Every 1 hour
65
		#Exec  file_cycle('c:\test\eventlog.txt', 2);
66
		#Exec  eventlog_out->reopen();
67
	#</Schedule>
68


69
# send log entries directly to InsightOps
70
 Module om_tcp
71
 Host ENDPOINT
72
 Port PORT
73
</Output>
74


75


76
<Route EventLog>
77
    Path        eventlog => eventlog_out
78
</Route>
79

Set up the event source

  1. Log in to InsightOps.
  2. Click Add Data in the top navigation.
  3. Click Quick add.
  4. Create a new log by selecting Token TCP.

Record the token and endpoint for later

Take note of the token and endpoint that are displayed when the log is created. You will need this information to configure NXLog in the next step.

Configure NXLog

  1. Replace the contents of the nxlog.conf file with the text in the sample configuration file.
  2. Replace the <LOG TOKEN GOES HERE> string with the log token you noted earlier.
  3. Replace ENDPOINT with the endpoint from the UI. For example, enter REGION.data.logs.insight.rapid7.com where REGION is your region code, such as eu, us, or ca.
  4. Replace PORT with 10000 for Token TCP.
  5. Restart the NXLog service by opening the services tool in the start menu. Search for NXLog in the services and then select Restart. This will restart NXLog and follow the new configuration.

Troubleshooting

If you find that the NXLog is not sending data, information can be found in the nxlog file at C:\\Program Files (x86)\\nxlog\\data\\nxlog.log.