Log Search Glossary
Familiarize yourself with the terms that are used in the Log Search documentation and UI.
Scope of this Glossary
The definitions provided in this topic are relevant exclusively to Log Search in InsightIDR. However, the same or similar terms may be used in other product areas and might have different meanings.
Quickly navigate through the glossary by clicking the first letter of the term that you want to learn about.
See Insight Agent.
A notification that informs you of notable events and suspicious behavior captured by InsightIDR. InsightIDR has built-in alerts, which are generated by UBA or ABA detection rules. However, you can also create basic detection rules (formerly known as custom alerts), which can be one of three types: Log Inactivity Detection Rules, Log Pattern Detection Rules, and Log Change Detection Rules. For more information, see Create and Manage Basic Detection Rules.
application programming interface (API)
A set of programming code that allows data transmission between one software product and another. For example, you can use the Log Search API to bulk query log sets, manage saved queries, and retrieve contextual log entries.
A single device on a network that can be identified by its IP address. In the Web interface and API, an asset may also be referred to as a device.
The process of mapping user accounts (for example, from Active Directory) to the endpoints where users log in. This makes it easier for analysts to investigate alerts by searching for the user who is involved, rather than by searching for an IP address or a host name.
In LEQL, clauses help you define your search criteria. Examples of popular clauses are
Rapid7 software that either polls data or receives data from event sources and makes it available for InsightIDR analysis. An event source represents a single device that sends logs to the Collector. By default, the Collector filters logs to cut down on duplicate or unnecessary data. The Collector sends the log data to the Insight Cloud for analysis.
In Log Search, the context menu allows you to build queries by selecting a clickable key or value from your search results. The context menu provides a set of contextually relevant operations so you can add to an existing query or create one from scratch.
Events provide insight about what is happening in your environment, such as user actions, system events, or errors. An event is typically recorded as a log entry—along with other events of the same type—in a log. Logs enter InsightIDR from one or more configured event sources.
Informational sources that you can connect to InsightIDR to provide visibility across your environment. Event sources can be networks, servers, firewalls, or anti-virus software. For example, if you have three firewalls in your environment, you will have one event source for each firewall.
A data structure that defines the data contained in an event. When event data comes into the InsightIDR system as logs, (such as from the Collector, event sources, sensors, or the Insight Agent) the application classes that event data as a particular type. For example, Firewall Activity.
LEQL functions help you perform operations on your log data to better understand your query results.
A LEQL clause that helps visualize your log data by grouping it by specified keys.
An Insight Agent is lightweight software installed on an endpoint to monitor the endpoint and report security-relevant events. The Insight Agent monitors specific event codes and collects endpoint telemetry data to provide an enhanced understanding of your endpoints' activity and drive quicker response time to detections. The agent collects data only from the asset on which it is installed.
In Log Search, your query results are divided into intervals when you add a
max function to your query. You can specify the number of intervals with the
timeslice(n) function, where n is a number between 1 and 200.
An Internet Protocol (IP) address is the unique number that is assigned to either a physical or virtual machine.
In Log Search, you can create basic detection rules that apply labels (also known as tags) to log entries to give a visual indication of the type of information they contain. For example, a warning label can be applied to log entries that contain one or more specific values that users should be warned about.
A Universally Unique Identifier (UUID) that is used to identify and manage a log in the UI and API, such as for setting its name.
A log set is a collection of multiple logs. In InsightIDR, a log set is defined (by default) by the type of event within the log stream, such as Firewall, DNS, Active Directory, and other event types. You can optionally create custom log sets in Data Collection or by using the API.
A Universally Unique Identifier (UUID) that is used to identify a log when sending data to that log in the Log Search platform service.
A loose search provides case-insensitive and partially matched results. Loose searches are used to query data when you don't know the case, or full string, of the value you're searching for.
Also referred to as a field, the key is a constant that defines the data in your logs. For example, the key could be
geoip_country_name and a value that pertains to that category might be,
United States. When logs are presented in the Table view in Log Search, the key names in the log become the column headings.
You can search for data in your logs by entering a combination of a key and any corresponding value. This is known as a key-value pair. For example,
geoip_country_name = United States. Note, that key-value searches support various operators, such as
CONTAINS. The operators that are supported depend on the type of key. LEQL supports strings, numbers, IP addresses (CIDR), lists, and regular expressions.
A keyword search allows you to find a string in any log, regardless of its format. Keyword searches are case-sensitive by default and will match a full string until it is delimited by a non-letter character.
InsightIDR supports both logical and comparison operators, which allow you to create more complex searches. Logical operators include
NOT. Comparison operators include
By default, Log Search orders your query results by newest ingestion time first. You can change the order of your logs by adding
sort(asc) to your query, or selecting the up arrow next to the query bar.
Parseable logs contain relevant security information derived from the more verbose data stream. These logs are parsed into key-value pair format to streamline data correlation across disparate events during incident response.
Data parsing is the process of taking data in one format and transforming it to another format. See also, unparsed data.
A precise request to retrieve information from within database and information systems. In Log Search, you type a query into the search bar to retrieve data from log entries.
A regular expression (regex) is a short-form query syntax that can be used along with LEQL to build queries in Log Search. Regular expressions use special characters as operators to allow you to search for more advanced patterns. InsightIDR supports the re2 version of regex.
Unmodified or unparsed data that is collected from a non-standard event source, such as a custom script or web application. Raw data is not attributed by InsightIDR. By default, raw data is stored in the
Raw Log log set.
A model or structure that organizes data into a specific format, so that the data can be read by the application.
A simple text or numerical value that can be used to search log data.
The partial or full contents of the
where() clause in a query. The search pattern is what you use to filter your data.
Structured logs are formatted as LEQL key-value pairs and can be read by humans and interpreted by machines, for easy searching, visualizing, or exporting.
In Log Search, the time range is the period of time your query results fall within.
In Log Search, the timeslice is the number of intervals your query results are divided into, or the unit of time they are divided by. InsightIDR calculates 10 equal time intervals when performing a query that uses a
average function. However, you can leverage the
timeslice function to manually set the number of intervals using either units of time (seconds, minutes, hours, days) or whole numbers. View the timeslice documentation.
In Log Search, a time span is the duration of the intervals that your log data is divided into on timelines.
Data that is being collected by event sources is often parsed, but it can also enter the application as unparsed data. The
Unparsed Data log set contains events that are collected from standard event sources that were not parsed as other types of activity. This behavior is enabled by selecting the Send Unparsed Data option when you configure an event source. You can view parsing information by going to the Event Source Health screen in InsightIDR. See also, parsed data.
Unstructured logs are strings of unpredictable text that cannot be parsed into key-value pair form and can be difficult to interpret or search. For this reason, Rapid7 supports automatic log structuring from known formats, such as CEF and JSON.
One of the preset roles in InsightIDR. It can also be defined in log data to identify the individual who was involved in log entries. You can investigate users in the Log Search screen or in the User Details screen.
Logs contain both keys, which are like fields, and values, which are like entries in a field. A value is the identifying piece of data, pertaining to a key, that you can search by. For example, if the key is
geoip_country_name, one of the possible values that pertains to that key might be
A variable is a placeholder in a LEQL query that represents one or more values.