Your connected event sources and environment systems produce data in the form of raw logs. Within InsightIDR, log data is categorized as log sets, logs, or log entries:
- A log set is a collection of multiple log streams. In InsightIDR, a log set is defined by the type of event within in the log stream, such as Firewall, DNS, Active Directory, and other event types.
- A log is a collection of log entries, or a single log stream. In InsightIDR, logs are typically named based on the source of the log stream. For example, Firewall: New York Office.
- A log entry is an individual log event.
Log Search takes every log of raw, collected data and automatically sorts them into Log Sets for you. New log events are added to the existing log and grouped with other logs in the log set based on when the log event was created. Once you apply a search to a log, a log set, or multiple log sets, you can do multiple things including:
- Search logs for specific terms with a Search Language.
- Build your own query to group by a field or calculate specific items.
- View logs in Visual Search.
- Create tags and Alerts on your log data.
- Export data to share with stakeholders.
Editing Log Streams and Log Events
Log data is stored in InsightIDR based on your retention policy. While users with Administrative access have the ability to permanently delete log sets, individual log events cannot be deleted or edited once they have been ingested by InsightIDR. InsightIDR may add enrichment data, such as geolocation or organizational data to events, however no original event data is ever removed or replaced. Events in InsightIDR reflect the events as they were originally generated by the source system.
Log View Options
When looking at the log entries, you can make reading logs easier by viewing log data in JSON format. Click Entries or Table to change views.
All normalized log entries can be queried either by searching for a string or by searching for a keyword=value pair.
While in Table View, you can quickly create a query based on a selected key, filter by column, or click on source_user and enter a username to search by, and run calculations.
Searching Your Data
InsightIDR allows users different ways of searching their data, including Regex, String, KeyValue, or Keyword search. See Use a Search Language for more information.
Or, you can build queries off of the provided Example Queries.
You can also build a query in Table View:
- While in Table view, select the keyword that you want to query on.
- Select an operator and enter a key value.
- Click Search.
- You can now search within your query.
You can export parseable logs to share with stakeholders at your convenience. When viewing the log entries table, select Export to CSV.
You will see a confirmation message appear. Your CSV file will be available in the Report Archive under the "Entries Export" tab.