Automatic Log Structuring

New Log Search is available for Open Preview

We are rolling out a new Log Search experience to customers with an open preview starting January 31st, 2023. You can still use original Log Search during this open preview. Both the original and New Log Search will exist in parallel until development is complete. For now, review the topic on new Log Search and navigate to the Log Search Open Preview page in InsightIDR to become familiar with the new layout. Check back soon for fully updated documentation.

When Automatic Log Structuring is enabled, InsightIDR will convert logs from known formats (such as CEF and JSON) into a human readable format, which allows you to write LEQL queries and search your logs with ease.

Without Automatic Log Structuring, InsightIDR encodes JSON or CEF log files as a string and places it in the source_data field of your log. This resulting source_data field reads as a single log line.

Since automatic log structuring is optional, log collection will continue as normal if you choose not to use it.

Before You Begin

If you want to take advantage of Automatic Log Structuring, make sure that you configure your appliance or third party device to send data in the CEF or JSON format.

If you have alerts, dashboards, or queries based on the source_data field, make sure to update them after enabling automatic structuring. Otherwise, they will become invalid.

Enable Automatic Log Structuring

To enable Automatic Log Structuring:

  1. Log in to InsightIDR.
  2. On the left menu, select the Settings page.
  3. At the bottom of the table, select the Automatic Log Structuring tab.
  4. Toggle on the Enable button.