Leverage Enhanced Endpoint Telemetry Data
While InsightIDR provides out-of-the-box detection rules for suspicious and malicious events, the information captured by the Insight Agent contains rich metadata that is useful for accelerating investigations and facilitating complete incident response. We refer to this data from the Insight Agent as Enhanced Endpoint Telemetry (EET) data, as it provides a more robust understanding of your endpoints’ activity.
Requirements
To access EET data, you need:
- A license for the Advanced or Ultimate packages, or access to the previously available EET add-on module. For pricing and packaging information, visit InsightIDR’s Packages and Subscriptions page.
- The Insight Agent installed on your endpoints.
Data retention
For InsightIDR Ultimate customers, EET data is available for 13 months. For InsightIDR Advanced customers, EET data is available for the previous 7 days only.
For details about log storage and retention in InsightIDR, view this solution brief .
Query EET data with the Endpoint Activity log set
You can view all of your EET data in Log Search using the Endpoint Activity log set. Run queries on this log set to analyze the activity relevant to your organization.
- In Log Search, select the Endpoint Activity log set.
- Update the time range as needed.
- Create a query:
- Refer to the sample queries to get started.
- Add the
loose()clause to ensure differences in capitalization don’t lead to missing query results. Read more about loose search.
- Optionally save your query so that you can use it again later.
Logs available in the Endpoint Activity log set
The Endpoint Activity log set contains three logs:
- Process Start - Contains events where a process starts on an endpoint.
- Netbios Poisoning - Contains events where protocol poisoning is seen on an asset.
- Local Service Creation - Contains events where any new services are installed on an asset, for example, PowerShell.
Sample Queries
Use these example queries to search your Endpoint Activity log data.
Replace values in bold from the example queries with values from your logs.
Query process start data
These queries are designed to help you find useful information in your Process Start data.
Groupby
There may be instances where you want to find data based on specific criteria. You can group your logs by domain, operating systems, file descriptions etc.
groupby(dns_domain)
groupby(os_type)
groupby(process.exe_file.description)
groupby(process.exe_file.product_name)Find all unique assets with chosen software running
where(process.name="**process.exe**" OR parent_process.name="**process.exe**")groupby(hostname)calculate(unique:hostname)limit(1000)Find software version
Parent Process
where(parent_process.name="**process.exe**")groupby(**parent_process.exe_file.version**)calculate(unique:hostname)Child Process
where(process.name="**process.exe**")groupby(**process.exe_file.version**)calculate(unique:hostname)Find hosts with psexec or psexecsvc running as either the parent or child process
where(process.name="**psexec.exe**" OR parent_process.name="**psexec.exe**")groupby(hostname)calculate(unique:hostname)limit(1000)
where(process.name="**psexecsvc.exe**" OR parent_process.name="**psexecsvc.exe**")groupby(hostname)calculate(unique:hostname)Find ping process by hostname and command line
where(process.name="**ping.exe**")groupby(hostname, process.cmd_line)Find Netstat processes by hostname and command line
where(process.name="**netstat.exe**")groupby(hostname, process.cmd_line)Find unsigned Windows processes
where(process.exe_file.signing_status.status="UNISGNED") broupby(process.name, hostname) calculate(count)Find processes that contain a specific word in their command line and/or by hostname
where(process.cmd_line icontains "**your word**")groupby(hostname, process.name)Find RDP by hostname and command line
where(process.name="**mstsc.exe**")groupby(hostname, process.cmd_line)Find Microsoft Management Console and its command line
where(process.name="**mmc.exe**")groupby(process.cmd_line)Find MMC launching ADUC by endpoint and username
where(process.name="**mmc.exe**" AND process.cmd_line icontains "**dsa.msc**")groupby(hostname, parent_process.username)Find Mimikatz by hostname and parent process command line
where("**mimikatz**", loose)groupby(hostname, parent_process.cmd_line)Group by processes running on unique hosts
groupby(process.name)calculate(unique:hostname)limit(1000)Find MSIExec installations
where(process.name="**msiexec.exe**" AND process.cmd_line icontains "**/i**")groupby(process.cmd_line)Find MSIExec quiet installations
where(process.name="**msiexec.exe**" AND process.cmd_line icontains "**/quiet**")groupby(process.cmd_line)Group by Linux process permissions and process name
where(os_type="**LINUX**")groupby(process.exe_file.permission, process.name)Group by process reputation
groupby(process.hash_reputation.reputation)Find all processes by name and by reputation
where(process.hash_reputation.reputation='Known')groupby(process.name)
where(process.hash_reputation.reputation='Unknown')groupby(process.name)
where(process.hash_reputation.reputation=’Malicious’)groupby(process.name)Find processes by reliability
where(process.hash_reputation.reputation='process_reputation')groupby(process.hash_reputation.reliability)Group by overall process reliability
groupby(process.hash_reputation.reliability)Find processes by reputational threat level
where(process.hash_reputation.reputation='process_reputation')groupby(process.hash_reputation.threat_level)Find processes by name, hostname, and username
where(process.hash_reputation.reputation='process_reputation')groupby(process.name, hostname, process.username)Find PUPs by process name and threat level
where(process.hash_reputation.classification.type='PUA')groupby(process.name, process.hash_reputation.threat_level)Find PUPs by name, hostname, and username
where(process.hash_reputation.classification.type='PUA')groupby(process.name, hostname, process.username)Find adware by process name
where(process.hash_repuation.classification.type='Adware')groupby(process.name)Group by process classification types
groupby(process.hash_reputation.classification.type)Find password documents
where(process.name=/(winword|excel|notepad|notepad++|textpad).exe/i AND process.cmd_line=/(.password.).(doc|txt|xls).*/i)groupby(process.cmd_line)Group by all process threat levels
groupby(process.hash_reputation.threat_level)Find low threat level processes by name
where(process.hash_reputation.threat_level='Low')groupby(process.name)Find processes by threat level
where(process.hash_reputation.reputation='process_reputation')groupby(process.hash_reputation.threat_level)Find processes with well-known reputation
where(process.hash_reputation.reliability IN ['Very high', 'High'])groupby(process.name)Find command lines showing the Taskkill.exe process
where(process.name="**taskkill.exe**")groupby(process.cmd_line)Find hostname and users using the PsLoggedon.exe utility
where(process.name="**PsLoggedon.exe**")groupby(hostname, process.username)Note: This query shows connected users on the local machine and remote connections using local endpoint resources.
Find hostname and users running the native screencapture.exe software
where(process.name='ScreenCapture.exe')groupby(hostname, process.username)Find processes, hostnames, and users running programs for SSH and/or Telnet
where(process.exe_file.description icontains-any ["**ssh**", "**telnet**"])groupby(process.name, hostname, process.username)Find hostnames and users performing the whoami command line
where(process.name="**whoami.exe**")groupby(hostname, process.username)Query process start data (Powershell)
These queries are designed to help you find useful information related to the PowerShell process.
Find Powershell processes
Parent Process
where(process.name="**powershell.exe**")groupby(parent_process.name)Child Process
where(parent_process.name="**powershell.exe**")groupby(process.name)Find Powershell process command lines that are not empty or are running a ps1 script
Parent Process
where(parent_process.name="**powershell.exe**" AND parent_process.cmd_line NOT IIN ["**null**", "**.ps1**"])groupby(parent_process.cmd_line) Child Process
where(process.name="**powershell.exe**" AND process.cmd_line NOT IIN ["**null**", "**.ps1**"])groupby(process.cmd_line)Find assets running powershell_ISE as either the parent or child process
where(process.name="**powershell_ise.exe**" OR parent_process.name="**powershell_ise.exe**")groupby(hostname)calculate(unique:hostname)Query process start data (PSEXEC)
These queries are designed to help you find useful information related to the PSEXEC process.
Find all psexec command lines
where(process.name="**psexec.exe**")groupby(process.cmd_line)Find psexec running the remote process in the system account
where(process.name="**psexec.exe**" AND process.cmd_line icontains "**.s.**")groupby(process.cmd_line)Find remote assets that Psexec is running processes under the system account
where(process.name="**psexec.exe**" AND process.cmd_line icontains "**.s.**" AND /psexec.exe \(?P<remote_asset>[^ ]*)/)groupby(remote_asset)Query historical user and asset data
When a suspicious event is detected on an endpoint, you can create a query to view all activity that occurred on the host prior to the event. You can use the queries in this section to display users and assets associated with a process start event.
Find unique assets associated with a process
where(hostname='**hostname**') groupby(process.cmd_line, process.exe_path) calculate(count)Find unique assets and users associated with a process
where(hostname='**hostname**' and process.username=’**username**') groupby(process.cmd_line, process.exe_path) calculate(count)Queries for Threat Hunting
These queries are designed to help you hunt for threats by analyzing endpoint user, process, and command line data.
Find commands being carried out by strings and stopping, all within the command line
where(process.name="**cmd.exe**" AND process.cmd_line ISTARTS-WITH "**cmd.exe /C**")groupby(process.cmd_line)Find command line attempts to remove certain folder attributes
where(process.cmd_line icontains "attrib -h -s -r")Find hosts and users that have launched a local endpoint’s User Account Management
where(process.name IIN ["**Netplwiz.exe**", "lusmgr.msc"])groupby(hostname, process.username)Find suspicious login activity
where(hostname="**hostname**" and process.username="**username**")Find additional information about a process that triggered a detection
where(process.pid='**process_id**' and hostname='**hostname**', loose)Find infrequently run commands
where(process.name='**process_name**' AND hostname='**hostname**') groupby(data.cmdLine) calculate(count) limit(**limit**)Note: Before running the sample query, replace process_name and hostname with the name of the process and host you want to group by.
Find malicious hashes
where(process.exe_file.hashes.sha1="**hash**") groupby(hostname) calculate(count)Find command lines where a setup.exe file is being launched from within the downloads folder
where(process.cmd_line icontains-all ["\downloads\","setup.exe"])groupby(process.cmd_line)Find commonly abused commands
Initial Investigation
where(process.cmd_line ISTARTS-WITH-ANY ["tasklist", "ver", "ipconfig", "systeminfo", "net time", "netstat", "whoami", "net start", "qprocess", "query"])groupby(hostname, process.cmd_line)Reconnaissance
where(process.cmd_line ISTARTS-WITH-ANY ["dir", "net view", "ping", "net use", "type", "net user", "net localgroup", "net group", "net config", "net share"])groupby(hostname, process.cmd_line)Spread of infection
where(process.cmd_line ISTARTS-WITH-ANY ["at", "reg", "wmic", "netsh advfirewall", "sc", "rundll32"])groupby(hostname, process.cmd_line)Understand the Enhanced Endpoint Telemetry metadata
This section provides the endpoint activity data that the Insight Agent sends to Log Search. You can create queries to group and detect on this data.
EET metadata varies by operating system
The keys that appear in Log Search vary based on the operating system of the device where the event occurred.
Process Start Event
The following table provides information about the process start event metadata collected by the Insight Agent. When you purchase the InsightIDR Ultimate package , you receive full access to the archive of process start data captured by the Insight Agent.
If All is listed in the Operating System column, the field is sent to Log Search regardless of the operating system of the device the event occurred on.
| Field | Description | Operating System |
|---|---|---|
| hostname | The hostname of the endpoint running the process. | All |
| dns_domain | The domain of the endpoint running the process. | All |
| os_type | The endpoint’s operating system. | All |
| r7_hostid | The Rapid7 Host ID. | All |
| process | All data related to the captured process. | All |
| parent_process | All data related to the process that spawned the started process. | All |
| env_vars | This object shows the environment variables when the process and its parent were launched. The parent value is listed only if it differs from the process value. The process value can be used to find processes that made changes to environment variables prior to launching a child process. If the env_vars of a process is null and the parent process includes environment variables, InsightIDR populates env_vars with the parent’s environment variables. | Windows |
| duplicated_events | The count of identical events that occurred in a process. This key is populated only if InsightIDR sees similar process events. | Linux |
Process Details
The Insight Agent collects and sends the following information about both the process triggering the event and the parent process.
Some fields vary based on the operating system that the process or executable file is running on. If All is listed in the Operating System column, the field is sent to Log Search regardless of operating system.
|Field |Description |Operating System | |---|---|---|---| |start_time | The time that this process started. |All | |name | The name of the process. |All | |pid | The system’s Process ID. |All | |ppid | The parent system’s process ID. |Mac/Linux, parent only | |r7_id | The Insight Agent-generated ID, unique to a process start. |All | |exe_path |The path to the executable. |All | |img_path | The path to the executable. This value might differ from exe_path if the executable is on a mounted remote file share. This key is sent to Log Search only if its value differs from exe_path. |Windows | |cmd_line |The command line invocation used to start the process, including arguments.|All | |username |The local user who started the process. |All | |account_domain |The AD domain of the user who started the process. |Windows | |uid |The user ID. |Mac/Linux | |group |The group name. |Mac/Linux | |gid |The group ID. |Mac/Linux | |euid_name |The effective user name. |Mac/Linux | |euid |The effective user ID. |Mac/Linux | |egid_name |The effective group name. |Mac/Linux | |egid |The effective group ID. |Mac/Linux | |ruid_name |The real user name. |Mac | |ruid |The real user ID. |Mac | |rgid_name |The real group name. |Mac | |rgid |The real group ID. |Mac | |fsuid |The file system user ID. |Linux | |fsgid |The file system group ID. |Linux | |suid |The saved user ID. | Linux | |sgid |The saved group ID. |Linux | |session |The login session ID that launched the process. |All | |addr |The remote address that the user is connecting from. |Mac | |port |The port the process used. |Mac | |exe_file |The information about the executable file. |All |
Executable File
The following table outlines the metadata that the Insight Agent collects from the executable file.
|Field |Description |Operating System | |---|---|---|---| |exe_file.owner |The owner of the executable file. |All | |exe_file.uid |The ID of the executable file owner. |Mac/Linux | |exe_file.group |The group of the executable file. |Mac/Linux | |exe_file.gid |The group ID of the executable file. |Mac/Linux | |exe_file.permissions |The permissions string of the executable file. |Mac/Linux | |exe_file.orig_filename |The original filename from the file metadata. | Windows | |exe_file.description |The description from the file metadata. |Windows | |exe_file.product_name |The product name of the executable, as reported by the file metadata. | Windows | |exe_file.author |The company who produced the executable, as reported by the file metadata. |Windows | |exe_file.version |The build version of the file, from the file metadata. |Windows | |exe_file.created |The executable file’s creation date. |All | |exe_file.last_modified |The executable file’s last modification date. |All | |exe_file.last_accessed |The executable file’s last accessed date. |Mac/Linux | |exe_file.size |The executable file’s size. |All | |exe_file.internal_name |The internal name of the executable file, from the metadata. | Windows | |hashes |The collection of different hashes of the process. |All | |signing_status |The signature status. |Windows | |signing_chain |The signature chain. |Windows | |countersigning_chain |The countersignature chain. |Windows |
Hashes
The Insight Agent collects and sends the following process hash information:
|Field |Description |Operating System | |---|---|---|---| |hashes.md5 |The MD5 hash. |All | |hashes.sha256 |The SHA256 hash. |All | |hashes.sha1 |The SHA1 hash. |All |