Rsyslog
Rsyslog is a syslog daemon commonly deployed in Debian and Ubuntu systems. It typically uses a simple TCP connection to send logs line-by-line. We support two methods of forwarding rsyslog events to InsightOps, which are explained below. We recommend using our Token-based input method which brings additional security and is independent of the actual source IP address.
/etc/rsyslog.conf
. Replace REGION_HERE with the region where your InsightOps account is located (e.g. eu, us, ca, au etc)
$template InsightOpsFormat,"TOKEN_HERE %HOSTNAME% %syslogtag%%msg%\n"
*.* @@REGION_HERE.data.logs.insight.rapid7.com:80;InsightOpsFormat
/etc/rsyslog.conf
. Replace REGION_HERE with the region where your InsightOps account is located (e.g. eu, us, ca, au etc)
*.* @@REGION_HERE.data.logs.insight.rapid7.com:PORT
Note: For UDP forwarding use a single @
. For encrypted TLS connections use @@REGION_HERE.data.logs.insight.rapid7.com
.
sudo service rsyslog restart
/var/log/myapp/errors.log
, you would enter the following lines in your /etc/rsyslog.conf
configuration file, above the previous snippet we added.
$Modload imfile
$InputFileName /var/log/myapp/errors.log
$InputFileTag myapp
$InputFileStateFile myapp-file1
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
# Only entered once in case of following multiple files
# This will poll the file every 10 seconds
$InputFilePollInterval 10
The setup tool allows you to automatically create a new Host and then for each file you wish to follow it will create a new log. The register documentation can be found here. Following a file is done by simply running sudo python le follow myfile
where myfile
is the path of the file you wish to follow.
More details can be found here. The setup tool can also read from a JSON configuration file which allows you to setup what files you wish to follow and what tokens to use. This is very useful is you are logging in an elastic enviroment. More details can be found here.
$InputFileTag
. Then you would enter the following in your /etc/rsyslog.conf
. Replace REGION_HERE with the region where your InsightOps account is located (e.g. eu, us, ca, au etc)
$template NginxTemplate,"TOKEN_HERE %HOSTNAME% %syslogtag%%msg%\n"
if $programname == 'mytag' then @@REGION_HERE.data.logs.insight.rapid7.com:80;NginxTemplate
& ~
You need to create a log in InsightOps and choose Token TCP and then the token thats printed in green is inserted above where it says TOKEN_HERE. You also need to make sure that the value you used for $InputFileTag
matches the value that $programname
is being checked against. The third line simply discards the event after its sent to InsightOps, you can omit this if you would like to also be sent somehwere else in your /etc/rsyslog.conf
.
rsyslog-gnutls
package. Sample configuration to enable encryption may look like this:
$DefaultNetstreamDriverCAFile /etc/ssl/certs/ca-certificates.crt
$ActionSendStreamDriver gtls
$ActionSendStreamDriverMode 1
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer *.data.logs.insight.rapid7.com
$template InsightOpsFormat,"LOG-TOKEN %HOSTNAME% %syslogtag%%msg%\n"
*.* @@REGION_HERE.data.logs.insight.rapid7.com:443;InsightOpsFormat
Note: you have to specify the right port number. For token based input, the port number is 443
.
Replace REGION_HERE with the region where your InsightOps account is located (e.g. eu, us, ca, au etc)
$InputFileTag: The tag to be used for messages that originate from this file. If you would like to see the colon after the tag, you need to specify it here (like ‘tag=”myTagValue:”’).
$InputFileStateFile: This is the name of this file’s state file.
$InputFileFacility: The syslog facility to be assigned to lines read. Can be specified in textual form (e.g. “local0”, “local1”, …) or as numbers (e.g. 128 for “local0”). Textual form is suggested. Default is “local0”.
$InputFileSeverity: The syslog severity to be assigned to lines read. Can be specified in textual form (e.g. “info”, “warning”, …) or as numbers (e.g. 4 for “info”). Textual form is suggested. Default is “notice”.
$InputRunFileMonitor: This activates the current monitor. It has no parameters. If you forget this directive, no file monitoring will take place.
$InputFilePollInterval: This setting specifies how often files are to be polled for new data. For obvious reasons, it has effect only if imfile is running in polling mode. The time specified is in seconds. During each polling interval, all files are processed in a round-robin fashion. A short poll interval provides more rapid message forwarding, but requires more system resources. While it is possible, we stongly recommend not to set the polling interval to 0 seconds. That will make rsyslogd become a CPU hog, taking up considerable resources. It is supported, however, for the few very unusual situations where this level may be needed. Even if you need quick response, 1 seconds should be well enough. Please note that imfile keeps reading files as long as there is any data in them. So a “polling sleep” will only happen when nothing is left to be processed.