Rsyslog Deployment via Chef

This document assumes you have a working knowledge of Chef, are using a Debian based server which already has Rsyslog. If not here is a good starting point to learn about Chef: The DevOps Tools We Use & How We Use Them

You will require an account on InsightOps, you can create a free account at InsightOps Free Trial.

It is assumed you already have Chef ecosystem running.

Using berkshelf or librarian­chef you will need to add the following to your Chef file or your Berksfile on your local machine.:

 
1
cookbook ‘le_rsyslog’, :git => 'https://github.com/joeheung/le_rsyslog'

Using a simple webserver for our use case where we setup nginx and rsyslog to log to logentries.com.

Our web server has 2 nginx logs:

 
1
/var/log/nginx/access.log
2
/var/log/nginx/error.log

Assuming you have a webserver cookbook that inherits nginx and our le_rsyslog cookbook we can override the below which is in your default atributes file.

 
1
node['le_rsyslog']['logentries']['logs'] 

The example would be to include the following in your web server attributes file.

 
1
override['le_rsyslog']['logentries']['logs'] =[ {:name => ’nginxaccess', :log => '/var/log/nginx/access.log', :token =>'01234567­0123­0123­0123­012345678901'}, {:name => ‘nginxerror', :log => '/var/log/nginx/error.log', :token => 'abcdefgh­abcd­abcd­abcd­abcdefghijkl'} ]

Now every time a web server is provisioned as a web server Rsyslog will automatically send their Nginx logs to logentries.com using the token specified in your attributes file.

With each token this will result in the log event being delivered to the associated log on logentries.

In logentries.com the logs will be formatted as follows:

 
1
webserverA nginxaccess 66.249.67.87 - - [20/Nov/2014:16:46:02 +0000] "GET /robots.txt HTTP/1.1" 301 178 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
2
webserverB nginxaccess 66.249.67.87 - - [20/Nov/2014:16:46:02 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
3
webserverA nginxaccess 70.115.128.218 - - [20/Nov/2014:16:53:10 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 6.3; Win64; x64; Trident/7.0; Touch; MAARJS; rv:11.0) like Gecko”
4
webserverB nginxaccess 190.64.75.50 - - [20/Nov/2014:16:18:31 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.36"
5
webserverB nginxaccess 71.80.102.213 - - [20/Nov/2014:16:18:46 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.111 Safari/537.36"
6
webserverA nginxaccess 195.224.113.195 - - [20/Nov/2014:16:29:51 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.111 Safari/537.36”