Leverage Enhanced Endpoint Telemetry Data
While InsightIDR provides out-of-the-box detection rules for suspicious and malicious events, the information captured by the Insight Agent contains rich metadata that is useful for accelerating investigations and facilitating complete incident response. We refer to this data from the Insight Agent as Enhanced Endpoint Telemetry (EET) data, as it provides a more robust understanding of your endpoints’ activity.
Requirements
To access EET data, you need:
- A license for the Advanced or Ultimate packages, or access to the previously available EET add-on module. For pricing and packaging information, visit InsightIDR's Packages and Subscriptions page.
- The Insight Agent installed on your endpoints.
Data retention
For InsightIDR Ultimate customers, EET data is available for 13 months. For InsightIDR Advanced customers, EET data is available for the previous 7 days only.
For details about log storage and retention in InsightIDR, view this solution brief.
Query EET data with the Endpoint Activity log set
You can view all of your EET data in Log Search using the Endpoint Activity log set. Run queries on this log set to analyze the activity relevant to your organization.
- In Log Search, select the Endpoint Activity log set.
- Update the time range as needed.
- Create a query:
- Refer to the sample queries to get started.
- Add the
loose()
clause to ensure differences in capitalization don't lead to missing query results. Read more about loose search.
- Optionally save your query so that you can use it again later.
Logs available in the Endpoint Activity log set
The Endpoint Activity log set contains three logs:
- Process Start - Contains events where a process starts on an endpoint.
- Netbios Poisoning - Contains events where protocol poisoning is seen on an asset.
- Local Service Creation - Contains events where any new services are installed on an asset, for example, PowerShell.
Sample Queries
Use these example queries to search your Endpoint Activity log data.
Replace values in bold from the example queries with values from your logs.
Query process start data
Groupby
There may be instances where you want to find data based on specific criteria. You can group your logs by domain, operating systems, file descriptions etc.
1groupby(dns_domain)2groupby(os_type)3groupby(process.exe_file.description)4groupby(process.exe_file.product_name)
Find all unique assets with chosen software running
1where(process.name="**process.exe**" OR parent_process.name="**process.exe**")groupby(hostname)calculate(unique:hostname)limit(1000)
Find software version
Parent Process
1where(parent_process.name="**process.exe**")groupby(**parent_process.exe_file.version**)calculate(unique:hostname)
Child Process
1where(process.name="**process.exe**")groupby(**process.exe_file.version**)calculate(unique:hostname)
Find hosts with psexec or psexecsvc running as either the parent or child process
1where(process.name="**psexec.exe**" OR parent_process.name="**psexec.exe**")groupby(hostname)calculate(unique:hostname)limit(1000)2where(process.name="**psexecsvc.exe**" OR parent_process.name="**psexecsvc.exe**")groupby(hostname)calculate(unique:hostname)
Find ping process by hostname and command line
1where(process.name="**ping.exe**")groupby(hostname, process.cmd_line)
Find Netstat processes by hostname and command line
1where(process.name="**netstat.exe**")groupby(hostname, process.cmd_line)
Find unsigned Windows processes
1where(process.exe_file.signing_status.status="UNISGNED") broupby(process.name, hostname) calculate(count)
Find processes that contain a specific word in their command line and/or by hostname
1where(process.cmd_line icontains "**your word**")groupby(hostname, process.name)
Find RDP by hostname and command line
1where(process.name="**mstsc.exe**")groupby(hostname, process.cmd_line)
Find Microsoft Management Console and its command line
1where(process.name="**mmc.exe**")groupby(process.cmd_line)
Find MMC launching ADUC by endpoint and username
1where(process.name="**mmc.exe**" AND process.cmd_line icontains "**dsa.msc**")groupby(hostname, parent_process.username)
Find Mimikatz by hostname and parent process command line
1where("**mimikatz**", loose)groupby(hostname, parent_process.cmd_line)
Group by processes running on unique hosts
1groupby(process.name)calculate(unique:hostname)limit(1000)
Find MSIExec installations
1where(process.name="**msiexec.exe**" AND process.cmd_line icontains "**/i**")groupby(process.cmd_line)
Find MSIExec quiet installations
1where(process.name="**msiexec.exe**" AND process.cmd_line icontains "**/quiet**")groupby(process.cmd_line)
Group by Linux process permissions and process name
1where(os_type="**LINUX**")groupby(process.exe_file.permission, process.name)
Group by process reputation
1groupby(process.hash_reputation.reputation)
Find all processes by name and by reputation
1where(process.hash_reputation.reputation='Known')groupby(process.name)2where(process.hash_reputation.reputation='Unknown')groupby(process.name)3where(process.hash_reputation.reputation=’Malicious’)groupby(process.name)
Find processes by reliability
1where(process.hash_reputation.reputation='process_reputation')groupby(process.hash_reputation.reliability)
Group by overall process reliability
1groupby(process.hash_reputation.reliability)
Find processes by reputational threat level
1where(process.hash_reputation.reputation='process_reputation')groupby(process.hash_reputation.threat_level)
Find processes by name, hostname, and username
1where(process.hash_reputation.reputation='process_reputation')groupby(process.name, hostname, process.username)
Find PUPs by process name and threat level
1where(process.hash_reputation.classification.type='PUA')groupby(process.name, process.hash_reputation.threat_level)
Find PUPs by name, hostname, and username
1where(process.hash_reputation.classification.type='PUA')groupby(process.name, hostname, process.username)
Find adware by process name
1where(process.hash_repuation.classification.type='Adware')groupby(process.name)
Group by process classification types
1groupby(process.hash_reputation.classification.type)
Find password documents
1where(process.name=/(winword|excel|notepad|notepad++|textpad).exe/i AND process.cmd_line=/(.password.).(doc|txt|xls).*/i)groupby(process.cmd_line)
Group by all process threat levels
1groupby(process.hash_reputation.threat_level)
Find low threat level processes by name
1where(process.hash_reputation.threat_level='Low')groupby(process.name)
Find processes by threat level
1where(process.hash_reputation.reputation='process_reputation')groupby(process.hash_reputation.threat_level)
Find processes with well-known reputation
1where(process.hash_reputation.reliability IN ['Very high', 'High'])groupby(process.name)
Find command lines showing the Taskkill.exe process
1where(process.name="**taskkill.exe**")groupby(process.cmd_line)
Find hostname and users using the PsLoggedon.exe utility
1where(process.name="**PsLoggedon.exe**")groupby(hostname, process.username)
Note: This query shows connected users on the local machine and remote connections using local endpoint resources.
Find hostname and users running the native screencapture.exe software
1where(process.name='ScreenCapture.exe')groupby(hostname, process.username)
Find processes, hostnames, and users running programs for SSH and/or Telnet
1where(process.exe_file.description icontains-any ["**ssh**", "**telnet**"])groupby(process.name, hostname, process.username)
Find hostnames and users performing the <code>whoami</code> command line
1where(process.name="**whoami.exe**")groupby(hostname, process.username)
Query process start data (Powershell)
These queries are designed to help you find useful information related to the PowerShell process.
Find Powershell processes
Parent Process
1where(process.name="**powershell.exe**")groupby(parent_process.name)
Child Process
1where(parent_process.name="**powershell.exe**")groupby(process.name)
Find Powershell process command lines that are not empty or are running a ps1 script
Parent Process
1where(parent_process.name="**powershell.exe**" AND parent_process.cmd_line NOT IIN ["**null**", "**.ps1**"])groupby(parent_process.cmd_line)
Child Process
1where(process.name="**powershell.exe**" AND process.cmd_line NOT IIN ["**null**", "**.ps1**"])groupby(process.cmd_line)
Find assets running powershell_ISE as either the parent or child process
1where(process.name="**powershell_ise.exe**" OR parent_process.name="**powershell_ise.exe**")groupby(hostname)calculate(unique:hostname)
Query process start data (PSEXEC)
These queries are designed to help you find useful information related to the PSEXEC process.
Find all psexec command lines
1where(process.name="**psexec.exe**")groupby(process.cmd_line)
Find psexec running the remote process in the system account
1where(process.name="**psexec.exe**" AND process.cmd_line icontains "**.s.**")groupby(process.cmd_line)
Find remote assets that Psexec is running processes under the system account
1where(process.name="**psexec.exe**" AND process.cmd_line icontains "**.s.**" AND /psexec.exe \(?P<remote_asset>[^ ]*)/)groupby(remote_asset)
Query historical user and asset data
Find unique assets associated with a process
1where(hostname='**hostname**') groupby(process.cmd_line, process.exe_path) calculate(count)
Find unique assets and users associated with a process
1where(hostname='**hostname**' and process.username=’**username**') groupby(process.cmd_line, process.exe_path) calculate(count)
Queries for Threat Hunting
These queries are designed to help you hunt for threats by analyzing endpoint user, process, and command line data.
Find commands being carried out by strings and stopping, all within the command line
1where(process.name="**cmd.exe**" AND process.cmd_line ISTARTS-WITH "**cmd.exe /C**")groupby(process.cmd_line)
Find command line attempts to remove certain folder attributes
1where(process.cmd_line icontains "attrib -h -s -r")
Find hosts and users that have launched a local endpoint's User Account Management
1where(process.name IIN ["**Netplwiz.exe**", "lusmgr.msc"])groupby(hostname, process.username)
Find suspicious login activity
1where(hostname="**hostname**" and process.username="**username**")
Find additional information about a process that triggered a detection
1where(process.pid='**process_id**' and hostname='**hostname**', loose)
Find infrequently run commands
1where(process.name='**process_name**' AND hostname='**hostname**') groupby(data.cmdLine) calculate(count) limit(**limit**)
Note: Before running the sample query, replace process_name and hostname with the name of the process and host you want to group by.
Find malicious hashes
1where(process.exe_file.hashes.sha1="**hash**") groupby(hostname) calculate(count)
Find command lines where a setup.exe file is being launched from within the downloads folder
1where(process.cmd_line icontains-all ["\downloads\","setup.exe"])groupby(process.cmd_line)
Find commonly abused commands
Initial Investigation
1where(process.cmd_line ISTARTS-WITH-ANY ["tasklist", "ver", "ipconfig", "systeminfo", "net time", "netstat", "whoami", "net start", "qprocess", "query"])groupby(hostname, process.cmd_line)
Reconnaissance
1where(process.cmd_line ISTARTS-WITH-ANY ["dir", "net view", "ping", "net use", "type", "net user", "net localgroup", "net group", "net config", "net share"])groupby(hostname, process.cmd_line)
Spread of infection
1where(process.cmd_line ISTARTS-WITH-ANY ["at", "reg", "wmic", "netsh advfirewall", "sc", "rundll32"])groupby(hostname, process.cmd_line)
Understand the Enhanced Endpoint Telemetry metadata
This section provides the endpoint activity data that the Insight Agent sends to Log Search. You can create queries to group and detect on this data.
EET metadata varies by operating system
The keys that appear in Log Search vary based on the operating system of the device where the event occurred.
Process Start Event
The following table provides information about the process start event metadata collected by the Insight Agent. When you purchase the InsightIDR Ultimate package, you receive full access to the archive of process start data captured by the Insight Agent.
If All
is listed in the Operating System column, the field is sent to Log Search regardless of the operating system of the device the event occurred on.
Field | Description | Operating System |
---|---|---|
hostname | The hostname of the endpoint running the process. | All |
dns_domain | The domain of the endpoint running the process. | All |
os_type | The endpoint's operating system. | All |
r7_hostid | The Rapid7 Host ID. | All |
process | All data related to the captured process. | All |
parent_process | All data related to the process that spawned the started process. | All |
env_vars | This object shows the environment variables when the process and its parent were launched. The parent value is listed only if it differs from the process value. The process value can be used to find processes that made changes to environment variables prior to launching a child process. If the env_vars of a process is null and the parent process includes environment variables, InsightIDR populates env_vars with the parent’s environment variables. | Windows |
duplicated_events | The count of identical events that occurred in a process. This key is populated only if InsightIDR sees similar process events. | Linux |
Process Details
The Insight Agent collects and sends the following information about both the process triggering the event and the parent process.
Some fields vary based on the operating system that the process or executable file is running on. If All
is listed in the Operating System column, the field is sent to Log Search regardless of operating system.
Field | Description | Operating System |
---|---|---|
start_time | The time that this process started. | All |
name | The name of the process. | All |
pid | The system's Process ID. | All |
ppid | The parent system's process ID. | Mac/Linux, parent only |
r7_id | The Insight Agent-generated ID, unique to a process start. | All |
exe_path | The path to the executable. | All |
img_path | The path to the executable. This value might differ from exe_path if the executable is on a mounted remote file share. This key is sent to Log Search only if its value differs from exe_path. | Windows |
cmd_line | The command line invocation used to start the process, including arguments. | All |
username | The local user who started the process. | All |
account_domain | The AD domain of the user who started the process. | Windows |
uid | The user ID. | Mac/Linux |
group | The group name. | Mac/Linux |
gid | The group ID. | Mac/Linux |
euid_name | The effective user name. | Mac/Linux |
euid | The effective user ID. | Mac/Linux |
egid_name | The effective group name. | Mac/Linux |
egid | The effective group ID. | Mac/Linux |
ruid_name | The real user name. | Mac |
ruid | The real user ID. | Mac |
rgid_name | The real group name. | Mac |
rgid | The real group ID. | Mac |
fsuid | The file system user ID. | Linux |
fsgid | The file system group ID. | Linux |
suid | The saved user ID. | Linux |
sgid | The saved group ID. | Linux |
session | The login session ID that launched the process. | All |
addr | The remote address that the user is connecting from. | Mac |
port | The port the process used. | Mac |
exe_file | The information about the executable file. | All |
Executable File
The following table outlines the metadata that the Insight Agent collects from the executable file.
Field | Description | Operating System |
---|---|---|
exe_file.owner | The owner of the executable file. | All |
exe_file.uid | The ID of the executable file owner. | Mac/Linux |
exe_file.group | The group of the executable file. | Mac/Linux |
exe_file.gid | The group ID of the executable file. | Mac/Linux |
exe_file.permissions | The permissions string of the executable file. | Mac/Linux |
exe_file.orig_filename | The original filename from the file metadata. | Windows |
exe_file.description | The description from the file metadata. | Windows |
exe_file.product_name | The product name of the executable, as reported by the file metadata. | Windows |
exe_file.author | The company who produced the executable, as reported by the file metadata. | Windows |
exe_file.version | The build version of the file, from the file metadata. | Windows |
exe_file.created | The executable file's creation date. | All |
exe_file.last_modified | The executable file's last modification date. | All |
exe_file.last_accessed | The executable file's last accessed date. | Mac/Linux |
exe_file.size | The executable file's size. | All |
exe_file.internal_name | The internal name of the executable file, from the metadata. | Windows |
hashes | The collection of different hashes of the process. | All |
signing_status | The signature status. | Windows |
signing_chain | The signature chain. | Windows |
countersigning_chain | The countersignature chain. | Windows |
Hashes
The Insight Agent collects and sends the following process hash information:
Field | Description | Operating System |
---|---|---|
hashes.md5 | The MD5 hash. | All |
hashes.sha256 | The SHA256 hash. | All |
hashes.sha1 | The SHA1 hash. | All |