InsightOps monitors a set of fields from the event source log.
- Source Address
- Public Suffix
- Top Private Domain
How to Collect DNS Server Logs
Perform the following steps on the server side for the InsightOps collector to incorporate logs from the DNS:
- Create a destination folder on the hard drive where the logs reside.
- Share that folder with a read-only credential that is also entered in InsightOps.
- Enable logging onto the service and direct those logons to the newly created folder.
Rapid7 recommends that the folder for DNS logging resides on the root (C) drive of the server that hosts the DNS, for example, C:\dnslogs.
Begin by creating the log file folder and sharing it:
- Create a folder for the DNS logs.
C:\dnslogsis the recommended directory for storing DNS logs.
- Right click the folder and select Properties from the drop-down menu. In the Properties dialog, click the Sharing tab and then click the Advanced Sharing button.
- In the Advanced Sharing dialog, select Share this folder and then click the Permissions button.
- In the Share Permissions dialog, click the Add… button and provide the credential that accesses this file. Include the user name and password for this credential in InsightOps when the DNS event source is set up.
- To enable logging onto the DNS server, right click the server’s name in the DNS Manager and select Properties from the drop-down menu.
- Click the Debug Logging tab, select Log packets for debugging, and enter the destination file name (the shared directory that you previously created in the File path and name field.) The remaining check boxes can keep the default values.
On the InsightOps side, you can configure the DNS event source to read the shared folder via UNC notation and by providing the credential that was used when setting up the shared folder. UNC notation is Microsoft's Universal Naming Convention which is a common syntax used to describe the location of a network resource.
NOTE: Make sure the file path includes the filename for the tail file as in the sample image. Unlike DHCP, just providing the directory path for the log is not sufficient for the DNS file configuration.
Configuring Microsoft DNS
InsightOps can collect Microsoft DNS audit logs.
To prepare to collect the logs, you need the DNS log to be written into a folder that the collector can connect to as a network share.
Microsoft DNS Servers Logs
Microsoft DHCP and DNS servers use similar technology to produce audit logs. In both cases, when logging is enabled, the services log their activity to a configured location on the file system. In order to read those logs in InsightOps, we provide file and directory watchers to automatically read in any changes to these log files. Share the folder that contains the log files in order to enable the collector to read these files over the network. This folder needs to be shared with a read-only credential that will also be provided to the DHCP and DNS event source configurations.
The Microsoft Domain Name Server (DNS) names resources that are connected to the Internet or a private network. It translates domain names, for example, www.mywebsite.com to its numerical Internet Protocol (IP) address, for example, 172.16.254.1. InsightOps can ingest these logs for further context around outbound traffic and network activity. DNS adds visibility, along with firewall, Web proxy, and other outbound traffic-based event sources, so that InsightOps can identify cloud services used by your organization. DNS logs are also available for detailed review in investigations.
Troubleshooting Configuration Issues
If the DNS event sources experience an error, the event source icon will turn to a yellow warning or red failure. Moving the mouse over the icon will reveal the details of the error. Typical errors of this sort are failure to connect to the server, bad credentials, or failure to find the file or folder configured in the event source.
Sometimes the DHCP and DNS event sources might not be reading any logs even if they don't show a warning or error. In this situation, try the following tests.
- Can you connect to the DHCP or DNS server file share when you log on to the machine running the InsightOps collector?
- Is there a typo in the file pattern in the DHCP configuration? If the file pattern is wrong, none of the files in the directory will match.
- Has srv.sys been set to start on demand on the server? Srv.sys should be set to start on demand. For more information, please read Srv.sys.
My Microsoft DNS Log File has 0 Bytes
It appears in some cases, whenever a log file needs to roll over, the old file cannot be deleted because the collector has it open. There is an article that discusses the issue here.
A temporary workaround for this issue is to enable DNS log file rotation and then use nxlog or a similar tool to collect the DNS logs, and forward them to the InsightOps collector as syslog.
Step 1: Enable DNS Log File Rotation
- Follow the instructions from the above sections to configure the Microsoft DNS server to create a single DNS text/debug log.
- After the log is created, on the DNS server, open a PowerShell command prompt as Administrator.
- Run the command:
Set-DNSServerDiagnostics -EnableLogFileRollover $true
- You can then verify that the DNS logging settings are correct by using the command:
What you expect to see is that the original dns.log in the same place it was created, but there is a new DNS log file with a timestamp inserted into the name.
The final configuration should look similar to this:
Step 2: Install and Configure Nxlog
- Follow the instructions from nxlog to install and use it to forward the DNS logs that you created above to the InsightOps collector.
Step 3: Set up the Event Source
- Configure the Microsoft DNS event source so that it is listening for syslog from the nxlog service.
Step 4: Enable Log File Deletion (Optional)
- You may wish to also enable the deletion of the old DNS logs so that they do not fill up the hard drive of the DNS server.
- Use the following command:
Get-ChildItem C:\locallogs\dnslogs | where LastWriteTime -lt ((Get-Date).AddDays(-2)) | Remove-Item -WhatIf
It has been at least 120 minutes since the last event.
The DNS event source sometimes can stop working and produce the above error.
However, the error is false because the dns log has not stopped logging. The log file can be opened from the collector, so there is no apparent reason for the error. A review of the collector.log may show the following error:
Read already in progress, consider increasing scan interval
Solution To fix this error and to allow the collector to read the file again, check the collector.log. Start at the bottom of the log and search upwards for the DNS server name, look for the following line:
FILE READ: smb://DNSServerNameHere/ShareName/dnsdebug.log [176748106 -> 176837156, Bytes Read: 89050]
If the file contains errors, an indication that the log is not being read, or the "read already in progress" messages, complete the following in order:
- Verify that an antivirus software has not locked the file. The folder where the log is located should be excluded from being scanning by AV software.
- When configuring debug logging on the DNS server, there is an option to configure a large file size before it can "roll over." If the file must becomes very large before it rolls over, decrease the log file size.
- Reboot the collector/restart the Rapid7 Collector service.
- Restart the DNS Server service.
- Reboot the DNS server.
- Delete the event source and recreate it.
Once the log is readable to the collector, you do not need to complete any additional steps. If the error persists, please contact Rapid7 for Support.