Search Your Logs

Once you configure Core Event Sources, go to the Log Search page from the InsightIDR homepage to search your logs, visualize your data, and create alerts.

Your connected event sources and environment systems produce data in the form of raw logs. Log Search takes every log of raw data and automatically sorts them into log sets for you.

On the Log Search page, go to the Log Sources panel and select the logs or log sets you want to search.

If you are unsure about which log sets to select, read about the Log Sets You Can Search.

To send data into Log Search in a format that is not currently supported by the Platform, you can set up custom logs.

To parse logs in a format that is unknown to InsightIDR, you can create custom parsing rules.

Step 2: Build a query

To search for specific log data, build a query using the Log Entries Query Language (LEQL). LEQL is the search language that allows you to build analytical queries. Try grouping your data using the groupby() clause to easily spot patterns in your results.

To begin learning the basics of LEQL, you can try recreating one of our example queries.

Step 3: View the resulting log entries

Depending on the type of query you run, the results display either in the Data tab, the Analysis tab, or both. A LEQL query that uses analytic functions displays its results in the Analysis tab.

The available LEQL analytic functions are:

  • Count: count
  • Sum: sum
  • Average: average
  • Count unique: unique
  • Minimum: min
  • Maximum: max
  • Timeslice: timeslice
  • Percentile: pctl
  • Bytes: bytes
  • Standard Deviation: standarddeviation

Tip for using the context menu

Select a clickable key or value to open the context menu. This menu allows you to quickly add a function, clause, key-value pair, or value to the query bar. By highlighting a specific value and using this menu, you can quickly enrich your queries and find specific data much more efficiently.

Reduce the number of groups returned when using the groupby() clause

The having() clause can be used to reduce the set of groups returned when using the groupby() clause. You can leverage calculation functions, such as sum, max, and unique within the having() clause to further specify the groups returned.

The having() clause is not supported in InsightIDR basic detection rules. Support for the having() clause in queries using a multi-groupby clause is coming soon.

Example groupby() query:
where(result=FAILED_BAD_PASSWORD) groupby(user)

Bad Password Query

This query includes all users who have failed to log in within the specified time range, including those users who only failed once.

Example groupby() query with the having clause:
where(result=FAILED_BAD_PASSWORD) groupby(user) having (count>5)

Bad Password Query with Having Clause

This query includes users who failed to log in more than 200 times. Any users who failed to log in less than 200 times will not be included in this query's results and could be interpreted as benign.

By default, Log Search orders your query results by most recent ingestion time first. Click the Order arrows to the right of the query bar to change the order of your query results. Log Search will persist your order selection for future logins.

Alternatively, you can use the sort() clause to change the order of your results:

  • sort(desc) or sort(descending)
  • sort(asc) or sort(ascending)

To sort by the name of the group instead of the value, include either sort(asc#key) or sort(desc#key) in your query.

If you are grouping by multiple keys, then you can pass in additional sorting criteria. Read more about multi-groupby queries.

Example

This query sorts the results first by the count of the destination_user key in ascending order, and then result key in descending order.

 
1
groupby(destination_user, result) calculate(count) sort(asc, desc#key)

Step 4: Create dashboards and visualizations (optional)

Create Dashboards for a holistic, visual understanding of your log data. Generate reports or export log data to share query results with your team.

Step 5: Create a Basic Detection Rule (optional)

Create a basic detection rule from specific log indicators, such as invalid logins.