Log Search Glossary

Familiarize yourself with the terms that are used in the Log Search documentation and UI.

Quickly navigate through the glossary by clicking the first letter of the term that you want to learn about.

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

A

agent

See Insight Agent.

application programming interface (API)

A set of programming code that allows data transmission between one software product and another. For example, you can use the Log Search API to bulk query log sets, manage saved queries, and retrieve contextual log entries.

asset

A single device on a network that can be identified by its IP address. In the Web interface and API, an asset may also be referred to as a device.

attribution

The process of mapping user accounts (for example, from Active Directory) to the endpoints where users log in. This makes it easier for analysts to investigate alerts by searching for the user who is involved, rather than by searching for an IP address or a host name.

C

clause

In LEQL, clauses help you define your search criteria. Examples of popular clauses are select(), where(), and groupby().

collector

Rapid7 software that either polls data or receives data from event sources and makes it available for InsightIDR analysis. An event source represents a single device that sends logs to the Collector. By default, the Collector filters logs to cut down on duplicate or unnecessary data. The Collector sends the log data to the Insight Cloud for analysis.

context menu

In Log Search, the context menu allows you to build queries by selecting a clickable key or value from your search results. The context menu provides a set of contextually relevant operations so you can add to an existing query or create one from scratch.

E

event

Events provide insight about what is happening in your environment, such as user actions, system events, or errors. An event is typically recorded as a log entry—along with other events of the same type—in a log. Logs enter InsightIDR from one or more configured event sources.

event source

Informational sources that you can connect to InsightIDR to provide visibility across your environment. Event sources can be networks, servers, firewalls, or anti-virus software. For example, if you have three firewalls in your environment, you will have one event source for each firewall.

event type

A data structure that defines the data contained in an event. When event data comes into the InsightIDR system as logs, (such as from the Collector, event sources, sensors, or the Insight Agent) the application classes that event data as a particular type. For example, Firewall Activity.

F

function

LEQL functions help you perform operations on your log data to better understand your query results.

G

groupby()

A LEQL clause that helps visualize your log data by grouping it by specified keys.

I

Insight Agent

An Insight Agent is lightweight software installed on an endpoint to monitor the endpoint and report security-relevant events. The Insight Agent monitors specific event codes and collects endpoint telemetry data to provide an enhanced understanding of your endpoints' activity and drive quicker response time to detections. The agent collects data only from the asset on which it is installed.

interval

In Log Search, your query results are divided into intervals when you add a count, average, min, or max function to your query. You can specify the number of intervals with the timeslice(n) function, where n is a number between 1 and 200.

IP address

An Internet Protocol (IP) address is the unique number that is assigned to either a physical or virtual machine.

K

key

Also referred to as a field, the key is a constant that defines the data in your logs. For example, the key could be geoip_country_name and a value that pertains to that category might be, United States. When logs are presented in the Table view in Log Search, the key names in the log become the column headings.

key-value pair

You can search for data in your logs by entering a combination of a key and any corresponding value. This is known as a key-value pair. For example, geoip_country_name = United States. Note, that key-value searches support various operators, such as =, !=, IN, CONTAINS. The operators that are supported depend on the type of key. LEQL supports strings, numbers, IP addresses (CIDR), lists, and regular expressions.

keyword search

A keyword search allows you to find a string in any log, regardless of its format. Keyword searches are case-sensitive by default and will match a full string until it is delimited by a non-letter character.

L

label

In Log Search, you can create basic detection rules that apply labels (also known as tags) to log entries to give a visual indication of the type of information they contain. For example, a warning label can be applied to log entries that contain one or more specific values that users should be warned about.

LEQL

Log Entry Query Language (LEQL) is a powerful search language that allows you to construct queries to extract the hidden data in your logs.

log

A collection of log entries that contain timestamped data about events. In InsightIDR, logs are typically named based on the event source. For example, Firewall: New York Office.

log entry

The data that is collected about an individual event, which is organized into keys and values. One log can contain hundreds or thousands of log entries.

log key

A Universally Unique Identifier (UUID) that is used to identify and manage a log in the UI and API, such as for setting its name.

log set

A log set is a collection of multiple logs. In InsightIDR, a log set is defined (by default) by the type of event within the log stream, such as Firewall, DNS, Active Directory, and other event types. You can optionally create custom log sets in Data Collection or by using the API.

log sources

The logs and log sets that act as the source of all of the data in Log Search.

log token

A Universally Unique Identifier (UUID) that is used to identify a log when sending data to that log in the Log Search platform service.

loose search

A loose search provides case-insensitive and partially matched results. Loose searches are used to query data when you don't know the case, or full string, of the value you're searching for.

O

operator

InsightIDR supports both logical and comparison operators, which allow you to create more complex searches. Logical operators include AND, OR, and NOT. Comparison operators include =, !=, IN, CONTAINS.

order

By default, Log Search orders your query results by newest ingestion time first. You can change the order of your logs by adding sort(asc) to your query, or selecting the up arrow next to the query bar.

P

parseable logs

Parseable logs contain relevant security information derived from the more verbose data stream. These logs are parsed into key-value pair format to streamline data correlation across disparate events during incident response.

parsed data

Data parsing is the process of taking data in one format and transforming it to another format. See also, unparsed data.

pre-computed queries

Pre-computed queries, a type of LEQL query, run continuously as log entries are received. They can be viewed immediately in either your Log Management settings or your custom dashboard cards. Pre-computed queries return results faster than a conventional query.

Q

query

A precise request to retrieve information from within database and information systems. In Log Search, you type a query into the search bar to retrieve data from log entries.

R

regular expression

A regular expression (regex) is a short-form query syntax that can be used along with LEQL to build queries in Log Search. Regular expressions use special characters as operators to allow you to search for more advanced patterns. InsightIDR supports the re2 version of regex.

raw data

Unmodified or unparsed data that is collected from a non-standard event source, such as a custom script or web application. Raw data is not attributed by InsightIDR. By default, raw data is stored in the Raw Log log set.

S

schema

A model or structure that organizes data into a specific format, so that the data can be read by the application.

search term

A simple text or numerical value that can be used to search log data.

search pattern

The partial or full contents of the where() clause in a query. The search pattern is what you use to filter your data.

structured logs

Structured logs are formatted as LEQL key-value pairs and can be read by humans and interpreted by machines, for easy searching, visualizing, or exporting.

T

time range

In Log Search, the time range is the period of time your query results fall within.

timeslice

In Log Search, the timeslice is the number of intervals your query results are divided into, or the unit of time they are divided by. InsightIDR calculates 10 equal time intervals when performing a query that uses a count, min, max, or average function. However, you can leverage the timeslice function to manually set the number of intervals using either units of time (seconds, minutes, hours, days) or whole numbers. View the timeslice documentation.

time span

In Log Search, a time span is the duration of the intervals that your log data is divided into on timelines.

U

unparsed data

Data that is being collected by event sources is often parsed, but it can also enter the application as unparsed data. The Unparsed Data log set contains events that are collected from standard event sources that were not parsed as other types of activity. This behavior is enabled by selecting the Send Unparsed Data option when you configure an event source. You can view parsing information by going to the Event Source Health screen in InsightIDR. See also, parsed data.

unstructured logs

Unstructured logs are strings of unpredictable text that cannot be parsed into key-value pair form and can be difficult to interpret or search. For this reason, Rapid7 automatically structures logs from known formats, such as CEF and JSON.

user

One of the preset roles in InsightIDR. It can also be defined in log data to identify the individual who was involved in log entries. You can investigate users in the Log Search screen or in the User Details screen.

V

value

Logs contain both keys, which are like fields, and values, which are like entries in a field. A value is the identifying piece of data, pertaining to a key, that you can search by. For example, if the key is geoip_country_name, one of the possible values that pertains to that key might be United States.

variable

A variable is a placeholder in a LEQL query that represents one or more values.