InsightIDR Ultimate | Quick Start Guide

As you get started, refer to this Quick Start Guide for guidance about how to approach InsightIDR's multi-step deployment process.

Not sure if you're in the right place?

If you purchased InsightIDR (not designated as Essential, Advanced, or Ultimate), please follow InsightIDR Quick Start Guide | Advanced for tasks and materials suited to your product.

InsightIDR Ultimate Flywheel

We’ve outlined the important deployment milestones in this recommended order:

  • Days 1-15: You'll install the Collector, deploy the Insight Agent, and set up six core event sources to begin aggregating your environment's data into InsightIDR and gain visibility across your network and digital perimeter.
  • Days 16-45: You'll review your detection coverage, explore investigations and Log Search, and begin to understand InsightIDR's deception technology so you can detect, investigate, and respond to threats efficiently and effectively.
  • Days 46-90: You'll configure basic detection rule (formerly known as custom alerts), create your first custom dashboard, and get started with automation connections and workflows to improve your detection coverage, alert fidelity, and incident response time.

Product Overview and Core Concepts

InsightIDR contains many features, but your first 90 days will center around the key parts of the product outlined in this section. For an overview of the areas you'll explore, watch our InsightIDR Demo video!

Here's a preview of the topics we cover in this demonstration:

  • 0:00 - Overview
  • 0:40 - Data Collection
  • 3:23 - Log Search
  • 4:29 - Dashboards and Reports
  • 5:51 - Detection Rules
  • 7:32 - Investigations
  • 8:58 - Automation
  • 10:11 - Settings
  • 11:07 - Conclusion
Core Components

On-premise components

Review InsightIDR's on-premises components to understand their purpose and value.

ComponentDescription
CollectorRequired for:
Event source log collection

To retrieve data, the Collector either polls event sources for the data, or has data pushed to it from the event sources. By default, the Collector filters logs to cut down on duplicate or unnecessary data. The Collector sends the log data to the Insight Cloud for analysis.

Each Collector must be installed on a dedicated host and only one Collector instance can be on each machine.
Insight AgentRequired for:
- Endpoint detection with detection rules and MITRE ATT&CK coverage
- Endpoint investigative telemetry
- Endpoint response activity
- Digital Forensics and Incident Response (DFIR) with Velociraptor

An Insight Agent is lightweight software installed on an endpoint to monitor the endpoint and report security-relevant events. To have the most cohesive understanding of your environment, it is recommended that you install an Insight Agent on all endpoints and servers you wish to monitor. The Insight Agent monitors specific event codes and collects endpoint telemetry data to provide an enhanced understanding of your endpoints' activity and drive quicker response time to detections. The agent only collects data from the asset on which it is installed.
Insight Network SensorRequired for:
- Network Traffic Analysis (NTA)
- Additional asset information such as, DNS queries, asset-connected countries and IP addresses, intrusion detection signatures, and more.

The Insight Network Sensor allows you to monitor, capture, and assess the end-to-end network traffic moving throughout your physical and virtual environments. InsightIDR uses network sensor data to generate investigations and alerts based on the network traffic traversing your environment, one of which is a new investigation data source based on IPv4 flow data. InsightIDR also leverages DNS and DHCP information that the network sensor extracts from network packets to produce other actionable alerts.
Insight OrchestratorRequired for:
- Automation workflows

The Insight Orchestrator is a server in your network or cloud environment that integrates your tools and systems with InsightConnect, Rapid7's security orchestration, automation and response (SOAR) solution.
Service AccountRequired for:
- User Behavior Analytics (UBA) powered by legacy detection rules

Service Accounts represent machine accounts that are typically designed for machine processing within your network. InsightIDR does not expect these accounts to be used outside your network. If you mark an account as a Service Account, InsightIDR will monitor for any activity entering your network using that account and will generate an INGRESS FROM SERVICE ACCOUNT alert to warn you of that activity. For example, InsightIDR would generate an alert if someone used a Service Account to log in to a VPN.
Event SourceRequired for:
- Specific detection rules and legacy detection rules

An event source is an application, appliance, server, service, or other IT asset that generates log events.

Detection rules, alerts, and deception technology

Review this section to understand InsightIDR's essential detection features, their purposes, values, and how they contribute to your environment's security.

ComponentDescription
Detection RulesDetection rules are the logic InsightIDR uses to detect threats using Rapid7’s wide array of threat intelligence. Our Detection Rule Library covers a variety of user behaviors and attacker techniques.

What it powers:
- MITRE ATT&CK matrix
- Alerts
Legacy Detection RulesLegacy detection rules run on InsightIDR's legacy User Behavior Analytics (UBA) engine. These rules apply insight to the millions of network events your users generate every day to detect compromised credentials, lateral movement, and other malicious behavior.

What it powers:
- Alerts generated by legacy detection rules
- Investigations generated by legacy detection rules
Basic Detection RulesBasic detection rules are available to you for the times when InsightIDR's built-in rules do not suit your needs or when you want to track certain behaviors that are specific to your environment. If you need to detect on certain, custom events, create a basic detection rule. There are three types of basic detection rules: Log Inactivity Detection Rules, Log Pattern Detection Rules, and Log Change Detection Rules.

What it powers:
- Basic detection rule investigations
- File Integrity Monitoring (FIM) investigations
Deception TechnologyDeception technology allows you to create an illusion for attackers that they have found something of interest in your environment. When you deploy intruder traps on your network, they act as a virtual trip wire. Once an attacker is tricked into accessing the trap, InsightIDR fires an alert to flag the suspicious activity.

What it powers:
- Legacy detection rules
- Investigations generated by legacy detection rules
HoneypotA honeypot is a virtual server that you can deploy on your network. Honeypots can look like any other machine on the network, or they can be deployed to look like something an attacker could target. You can have a single honeypot or multiple honeypots, and you can deploy them straight out of InsightIDR. Honeypots lie in wait for "attacker" events to happen, such as a port scan or an attempted user authentication, which immediately sets off an alarm. If you deploy the Rapid7 Honeypot and enable the associated alerts in InsightIDR, you will be notified if such an activity occurs. Once attackers find an initial foothold in a network, their next step is typically to run a network scan to identify all the other assets in the network.

Includes Honey Users, Honey Files, Honey Credentials, and Honey Alerts

What it powers:
- Honeypot Access detection rule and investigations
- Honey User Authentication detection rule and investigations
- Honey File Accessed detection rule and investigations
- Local Honey Credential Privilege Escalation Attempt detection rule and investigations
- Remote Honey Credential Authentication Attempt detection rule and investigations
- Corresponding Honey Alert type investigations
File Integrity Monitoring (FIM)File Integrity Monitoring (FIM) allows you to audit and monitor changes to critical files and folders on your network. When you turn on FIM, the Insight Agent starts collecting FIM events. InsightIDR can then attribute users to file modification activity. You can create alerts based on certain file log events to notify you when one of your users modifies a critical file or folder.

What it powers:
- FIM alerts
- FIM investigations
InsightIDR Architecture

Prepare for Deployment

To ensure you get the most out of your first 90 days with InsightIDR, it’s important to understand your deployment tasks and create a plan for deployment.

Supported Browsers

Rapid7 supports InsightIDR in Google Chrome (latest stable release) and Mozilla Firefox (latest stable release).

Set up a service account

Before provisioning resources and deploying InsightIDR, you must set up a Service Account to collect log data for InsightIDR. You can either designate an existing user account, or create a Service Account.

Review the Service Account requirements and set up an account.

Review system requirements

Each component of InsightIDR requires specific resources. Navigate to each component's designated page for a full list of requirements.

ComponentRequired Resources
Collector- 4 CPU cores with 2GHz+ on each core
- 8 GB RAM recommended
- 60 GB+ available disk space
- Configured with a Fully Qualified Domain Name (FQDN) such as idrcollector23.myorg.com

See Collector Requirements requirements for more information.
Insight AgentRequired ports for Collector communication through TCP:

- 5508
- 6608
- 8037

If you are using the Collector for endpoint monitoring, ensure the following ports are open:

- 5508
- 6608
- 20000 – 30000

See Insight Agent Requirements requirements for more information.
Service Account<You can either designate an existing user account as a service account, or create a new account account as your service account, that meets all of the following requirements:

- Active Directory Permissions
- LDAP Permissions
- Microsoft DNS Permissions
- Microsoft DHCP Account Permissions
- Endpoint Monitor

See Setting Up a Service Account for more information.
Event SourceDesignate a Service Account with the correct permissions

See Core Event Sources for more individual event source requirements.
Honeypot- 1 CPU
- 1GB RAM
- 10 GB hard disk space

See Honeypot Requirements for more information.
Insight Network SensorAlthough the network sensor software itself runs in the form of a container, all physical or virtual network sensor hosts must run one of the following supported Linux operating systems. The version number shown for each one indicates the minimum supported version:

- Ubuntu Server 20.04 and later
- RHEL 7.2 and later
- CentOS 8 and later
- Fedora 30 and later
- SUSE 15.0 and later
- Debian 8.11 and later

See Network Sensor Host Requirements for more information.
Provision resources in your environment

We recommend provisioning specific resources as soon as possible to ensure a quick and easy deployment experience.

These core components require provisioning hardware:

Log in to the Insight Platform

Already have an Insight Platform account?

If you already have a platform account from a trial or existing subscription to another Rapid7 solution, you’re all set! Use your existing email address to log in to https://insight.rapid7.com/login.

The Rapid7 Insight Platform is your base within the ecosystem of Rapid7 cloud products and services. It provides a centralized location for administrative functions and makes navigating the Insight product suite simple and easy.

To log in to the platform, you need a Rapid7 Insight Platform account.

To create an account:

  1. Check your corporate email inbox for an email from the Rapid7 Insight Platform team.
  2. Visit insight.rapid7.com/login.
  3. Select Haven’t activated your account?.
  4. Enter your corporate email address to receive an activation email with next steps. If you do not receive an activation email, reach out to your Customer Adoption Manager (CAM) or Customer Success Manager (CSM).
  5. Refer to the activation email and follow the instructions to create and activate your Insight Platform account
Configure daily data archiving

InsightIDR stores your log data for a retention period of 13 months. If you need to retain data for longer than that period, such as for security investigation or compliance purposes, we recommend that you set up daily archiving. Archiving allows you to retain a copy of your log data using the storage capabilities of Amazon S3.

To set up data archiving, see Data Archiving.

Daily Archiving versus Historical Data Archiving

If you do not configure daily archiving, you can download a backup of your data up to 2 times a year using InsightIDR's Historical Data Archiving feature. This process can take several days to complete.

Additional Resources

Days 1 - 15: Get Up and Running

Install Collectors, deploy Insight Agents, and set up core event sources so InsightIDR can begin collecting the right data to detect, investigate and respond to threats in your environment.

Install the Collector

The Collector is the on-premises component of InsightIDR that either polls data or receives data from Event Sources and makes it available for InsightIDR to analyze. (An Event Source represents a single device that sends data in the form of logs to the Collector.)

The Collector captures the data generated by your event sources, compresses the data, encrypts it, and pushes it up to the Insight Platform. The Insight Platform will then normalize, attribute, analyze, and present that data for search.

How to install

Identify all the servers where event source data originates from. Many networks consolidate security and network administration tools and services in a data center or corporate office. This central location is an ideal place to deploy a Collector on a dedicated host.

  1. Review the Collector Requirements.
  2. Install collectors on network servers or virtual machines that meet the following requirements:
    • 4 CPU cores with 2GHz+ on each core
    • 8 GB RAM recommended
    • 60 GB+ available disk space
    • Configured with a Fully Qualified Domain Name (FQDN) such as idrcollector23.myorg.com

For detailed setup instructions, see Collector Installation and Deployment.

Deploy the Insight Agent

The Insight Agent enables InsightIDR to provide continuous endpoint detection and response, which is necessary for identifying the early signs of an attack.

The Insight Agent collects live system information – including basic asset identification information, running processes, and logs – from your assets and sends it to the Insight Platform for analysis. The data is then passed to InsightIDR, where it unlocks the vast library of attacker behavior and endpoint detection rules. Combined with network and user data, this helps you identify stealthy attacks for both your critical and remote assets across the MITRE ATT&CK framework.

How to deploy the Insight Agent

Each Insight Agent only collects data from the endpoint on which it is installed. You must install the Agent on all assets you want InsightIDR to monitor. The Insight Agent can be installed directly on Windows, Linux, or Mac assets.

You can deploy the Insight Agent on your target assets using either a token-based installer or certificate package installer. The token-based installer uses a token to download configuration files, whereas the certificate package installer provides a .zip file with the configuration files.

  1. Decide which installer is best suited for your environment.
  2. Read the steps to download the installer and follow the instructions.
Set up core event sources

InsightIDR’s core event sources provide the most data in regards to user attribution. User attribution correlates endpoint activity to individual users, including what endpoint applications they use, and when. Attribution gives you a more complete image of your security posture, as user accounts are the most common targets for sophisticated attacks.

Setup instructions for each event source can vary depending upon the technologies you use.

To take advantage of InsightIDR's detection rule capabilities, you must set up these event sources:

LDAP

Required for: Detection Rules and Legacy Detection Rules

Adding a Lightweight Directory Access Protocol (LDAP) server allows InsightIDR to track the users, admins, and security groups contained in the domain and to link account activity with real users to identify privileged and service accounts.

To add the LDAP event source:

  • Designate a Service Account with the correct permissions.
  • Open Port 636 (LDAPS) between the Collector and the LDAP server.

For detailed setup instructions, see LDAP.

Active Directory

Required for: Detection Rules and Legacy Detection Rules

Active Directory provides security logs from your domain controllers and authentication and administrative events for your domain users. Make sure that you add one Active Directory event source for each domain controller.

To add the Active Directory event source:

  • Open ports 135, 139, and 445 between the Collector and Active Directory.
  • Designate a Service Account with the correct permissions.

For detailed setup instructions, see Active Directory.

DHCP

Required for: Legacy Detection Rules

Dynamic Host Configuration Protocol (DHCP) event logs provide IP lease information to correlate each IP address with its assigned host at the time of the event.

To add the DHCP event source:

Choose one of the following methods:

  • Domain Admin Account: Use this method if you want to collect logs using a Domain Admin account.
  • NXLog: Use this method if you don’t want to use a Domain Admin account to collect logs.
DNS

Required for: Legacy Detection Rules

Connecting DNS as an event source allows InsightIDR to track services, incidents, and threats found on your network. DNS logs gathered from a DNS event source provide more information about web traffic. DNS also provides greater visibility into destination URLs, which can be flagged in Account Visited Suspicious Link incidents.

For instructions on how to configure DNS appliances with InsightIDR, see DNS.

VPN

Required for: Detection Rules and Legacy Detection Rules

VPN logs provide visibility into users' remote network ingress activity and allow you to collect and verify information about user activity.

For setup instructions, see VPN.

Firewall

Required for: Detection Rules

By introducing Firewall data, you allow InsightIDR to track visits to malicious domains or cloud services. More than simply collecting configuration logs and change logs, InsightIDR can automatically attribute connection events to the users and endpoints that are accessing such websites.

We recommend you set up separate Firewall event sources for each Firewall type.

To add a Firewall event source:

Determine which Firewall event source(s) best suits your needs and follow the related configuration steps.

For setup instructions for each type of Firewall event source, see Firewall.

Days 16 - 45: Alert and Investigate

Dive into InsightIDR’s detection coverage! Explore detection rules, investigations, alerts, deception technology, and Log Search to see how all of these components work together. Start using InsightIDR's powerful query capabilities, and visualize your log data using dashboards.

Review your detection coverage

InsightIDR Detection Engines

InsightIDR's detection capability is powered by 3 detection engines – The detection rule engine, the legacy UBA engine, and the basic detection rule engine (formerly known as custom alerts). Let's take a look at each:

  • Detection rule engine: The primary detection rule engine detects on a variety of user behaviors and attacker techniques. Each detection rule hunts for a unique attacker behavior or user behavior.
  • Legacy UBA engine: The legacy UBA engine is powered by built-in legacy detection rules that apply insight to the millions of network events your users generate every day. These rules detect compromised credentials, lateral movement, and other malicious behavior.
  • Basic detection rules: InsightIDR provides powerful detection capabilities, but there may be times when they do not cover all your requirements. The basic detection rules engine allows you to create custom alerting rules to detect inactivity, patterns in data, or changes in data.

Detection Rules

Detection rules are the logic InsightIDR uses to detect threats using Rapid7’s wide array of threat intelligence. When the logic of a detection rule matches an event, an alert is created. Certain alerts automatically trigger investigations – this behavior is defined in the detection rule logic. On the Detection Rules page, you will find 2 tabs of built-in detection rules:

  • Detection Rule Library: The Detection Rule Library contains rules that detect on a variety of user behaviors and attacker techniques. You can see all detection rules and the number of detections over the last 30 days on the Detection Rule Library tab of the Detection Rules page. You can also view rule logic, related Threat Groups and MITRE ATT&CK mapping, and filter, sort and search detection rules.
  • Legacy Detection Rules: Legacy Detection Rules run on InsightIDR’s legacy User Behavior Analytics (UBA) engine. These rules apply insight to the millions of network events your users generate every day to detect compromised credentials, lateral movement, and other malicious behavior. To view legacy detection rules in InsightIDR, click Detection Rules in the left menu, and click the Legacy Detection Rules tab.

All out-of-the-box detection rules can be tuned to suit your environment's needs:

Notable Events and Alerts that allow you to prioritize your response

Every moment InsightIDR is running, it finds and collects thousands of threats from all the assets and endpoints it has visibility into. However, many activities set off alerts even though they are not high profile threats, causing unnecessary alarm and risking potential burnout for Security teams investigating these alerts.

Notable Events
As a part of InsightIDR's legacy user behavior analytics (UBA), InsightIDR monitors and baselines each user's activity, establishing a pattern for which assets the user owns, which assets they access, their remote login location, and other activity. Notable Events are the events that fall between expected behavior and alert-worthy behavior. It is expected that almost any network will experience a large number of notable events on any given day.

When user activity triggers an alert that generates an investigation, related notable events are automatically added to Investigation Timeline. This helps analysts better contextualize the behavior without these notable events going completely unobserved, and ensures your security team isn’t overwhelmed with low fidelity alerts.

Alerts
Alerts arrive to you in the form of an email with an incident number and short description to inform you of notable events and suspicious behavior captured by InsightIDR. There are a few different types of alerts and alert settings available:

Explore investigations
Investigations are an aggregate of the applicable alert data in a single place and are closely tied to Detection Rules and Alerts. The Investigations page allows you to quickly search your active investigations and provides you with the context you need to effectively prioritize, sort, and respond to alerts.

InsightIDR Investigations Page Preview

Let's review the Investigations home screen:

  1. Investigations are listed in order of priority -- low, medium, high, and critical. You can also sort them by date created and by most recent detection.
  2. Filter your investigations by their attributes.
  3. Select the date range of the investigations you want to explore.
  4. Search your investigations by investigation name, associated users, or associated assets.
  5. Create a new investigation.
  6. Toggle between timeline and table view for a detailed look into a specific investigation.
  7. Change the status of an investigation to open, investigating, or closed.

See Investigations to learn more.

Explore Log Search and Queries

Your connected event sources and environment systems produce data in the form of raw logs. Log Search takes every log of raw, collected data and automatically sorts them into log sets for you. New log events are added to the existing log and grouped with other logs in the log set, based on when the log event was created.

When data is ingested, repetitive activity is processed and combined into a single entry. The count of the repeated events are included in the combined entry, and other fields such as bytes transferred are summarized. To learn more about data deduplication, read Collector Overview.

Use queries to enrich alerts and investigations

You can search your logs to learn more about the incident in your environment that triggered the alert or the investigation.

Log Entry Query Language (LEQL) is used when building queries in InsightIDR. LEQL supports Regular Expressions (RegEx) and can be used when creating basic detection rule queries and dashboards.

Log Search supports many InsightIDR capabilities:

  • Data ingestion
  • Detection
  • Basic detection rules
  • Dashboards and reporting
  • Compliance standards

Build and save a query

InsightIDR provides different ways to search your data, including RegEx, String, KeyValue, or Keyword searches. With LEQL, you can construct queries to extract the hidden data within your logs.

LEQL follows SQL-style syntax and constructing a query is simple and intuitive. InsightIDR requires you to input a where() clause first (if your query requires one) and add subsequent clause(s) and function(s), such as groupby() or calculate, respectively.

  • where () = search
  • groupby () = field
  • calculate (:) = function:field

Try it out

Test these queries in Log Search and see the results for yourself.

  • where(result ISTARTS-WITH "failed" AND logon_type!=NETWORK)groupby(destination_user, destination_asset)limit(5, 1)
  • where(action=MEMBER_ADDED_TO_SECURITY_GROUP AND group IIN [admin, desktop, network, support, "service accounts"])groupby(source_user, target_account, group)
  • where(source_account==target_account)groupby(action, source_user))

To save a query for future use, click the ellipsis next to the Run button and select Save Query.

See Log Search for more information.

Visualize your data with dashboards

Dashboards allow you to build custom views of the data you want to monitor. As a starting point, either create a new dashboard or use an existing dashboard. Add, edit, resize, and rearrange the data visualization cards to tailor the data view to your needs.

Explore the IT Operations Dashboard

The IT Operations Dashboard offers a high level view of Weekly Account Lockouts and Unlocks, Most Active admin accounts on user accounts. Most Active admin accounts on their own accounts, Top 5 Directory groups by most membership adds, and more.

The IT Operations Dashboard is populated by data sent from Active Directory, Firewall, VPN, and Cloud Service event sources. To take advantage of this dashboard, be sure those event sources have been set up.

  • Active Directory event sources send Active Directory Admin Activity, Asset Authentication and Raw log data.
  • Firewall and VPN event sources send Firewall Activity and Ingress Authentication logs.
  • Cloud Service event source send Ingress Authentication logs.

To set up the IT Operations Dashboard

  1. From the InsightIDR left menu, click Dashboards and Reports.
  2. From the dashboard list, click the Dashboard Library button. The Dashboard Library modal appears.
  3. Search for the IT Operations Dashboard and click Add.
Understand deception technology

Some types of stealthy behavior can be difficult to discern from normal activity, allowing the attacker to sneak past your security tools and into your organization undetected. Placing intruder traps as a way to distract attackers can often help you find attackers earlier and take action to block them before they access something they should not.

Review Honey Items

Rapid7 Honey Items, a type of intruder trap, can attract hackers because they are “sweet” with opportunity. The most common type of intruder trap is a honeypot, which is a decoy system designed to gather information about attackers on your network. It can help you learn how attackers are accessing your systems.

InsightIDR has four kinds of deception traps that you can configure and deploy. Attempted access or use of these intruder traps triggers Honey Alerts in InsightIDR. We recommend deploying deception traps in this recommended order:

  1. Honeypots: Deploy these to cover as much network as possible (for instance, one for each subnet).
  2. Honey Users: Add a user to the Active Directory that matches how you typically configure usernames, but also conveys it may be a domain admin.
  3. Honey Files: Deploy files that appear to be valuable, such as a financial report or something with personally identifiable information (PII).
  4. Honey Credentials: These are managed by Rapid7 on the Agent so no setup is required.

Configure a honeypot

A honeypot is a virtual server that you can deploy on your network from InsightIDR. Honeypots can look like any other machine on the network, or they can be deployed to look like something an attacker could target. You can have a single honeypot or multiple honeypots, and you can deploy them straight out of InsightIDR. Honeypots lie in wait for “attacker” events to happen, such as a port scan or attempted user authentication, which immediately sets off an alarm. If you deploy the Rapid7 Honeypot and enable the associated detection rules in InsightIDR, you will be notified if such activity occurs. Once attackers find an initial foothold in a network, their next step is typically a network scan to identify all the other assets in the network.

InsightIDR's Honeypot is an OVA appliance designed for deployment in VMware environments. The honeypot can detect network reconnaissance, typically in the form of suspicious network or port scanning. The Honeypot OVA contains an appliance that is able to listen on all ports. All scanning or connection attempts are allowed. Each time a connection is attempted, the honeypot captures information about the source asset, and potential user associated with the connection. This data is immediately pushed up to the Insight platform, generating a Honeypot Access Alert.

How to configure a honeypot

  1. From the InsightIDR homepage, click Data Collection on the left menu.
  2. In the top right corner, click Setup Honeypot and select Download Honeypot.

The Honeypot OVA download will begin automatically. See Download a Honeypot for next steps.

Additional Resources

Days 46 - 90: Optimize for Success

Configure basic detection rules (formerly known as custom alerts), create custom dashboards, and create automation workflows to tailor InsightIDR to your environment's unique needs.

Configure Basic Detection Rules

With InsightIDR, you have the option of creating basic detection rules when built-in rules do not suit your needs. You configure basic detection rules in Log Search. Basic detection rules can be created manually, or configured to auto-populate. You can always switch to a different rule type during configuration. There are three kinds of basic detection rules.

Log Inactivity Detection Rules

Also known as “Up Down Monitoring”, inactivity rules can be used to notify you when an entire log, log group, or particular pattern becomes inactive for a given time period.

For setup instructions, see Auto-populate a Log Inactivity Detection Rule or Manually configure a Log Inactivity Detection Rule.

Log Pattern Detection Rules

In order for a detection to trigger, a log must match the exact pattern you enter as a search term. Detecting on patterns can be useful in situations such as monitoring server errors, critical exceptions, and general performance, and allows you to only monitor events that are important to you.

For setup instructions, see Auto-populate a Log Pattern Detection Rule or Manually create a Log Pattern Detection Rule.

Log Change Detection Rules

Log change detection rules will notify you when a condition changes, such as HTTP 500 errors in your web access logs. They are based on calculations that you apply to logs or log sets.

For setup instructions, see Auto-populate a log Change Detection Rule or Manually configure a Log Change Detection Rule.

Customize your dashboards

InsightIDR's dashboards are customizable to ensure your views of your environment meet your needs. While it's easier to customize a dashboard template, you can also create a new one from scratch. Although dashboard customization is up to you, we'd like to share some key elements to explore while building your dashboards.

Edit the cards

After adding a card to your dashboard, click the gear icon in the top right corner of a card and select Edit.

  1. Name Card : Change the card's name and description to provide more clarity from the dashboard view.
  2. Select Data Source :
    • Use a previously saved query and customize the Full Query to show the data you want to highlight.
    • Change the time range for the data shown in the card.
    • Change the log set the card's data is referencing.
  3. Configure Chart : Add a caption, add data labels, change the scale, and change the colors of your card
    • Captions are a free text form to provide additional context of the card's data from the dashboard view.
    • Colors are best for differentiating between multiple groups in one card. Try editing the colors of a Stacked Bar chart!
  4. Switch Charts : You can choose the best way to visualize your card's data from the charts listed in the Switch Charts section. InsightIDR will only make the charts available that can correctly display your data.

Apply a LEQL query to an entire dashboard

By clicking the filter icon in the top menu of your dashboard, you can add a query to filter all of your dashboard's data. Check out our documentation for help building a query.

Choosing the right chart for your data

To determine the best chart to display your data, here are some example use cases for each format.

  • For examining trends or activity over time, use an Area, Line, Stacked Area, or Stacked Bar chart.
  • For comparisons across different categories, use a Bar or Bar Horizontal chart.
  • For understanding top trends and groups, use a Packed Bubble, Word Cloud, or Pie chart.
  • For a quick glance at a single data point, use a Number chart.
  • For visualizing cyclical data in relation to time, use a Radial chart.
  • For an easy-to-scan view of individual data points across categories, use a Table chart.

Size your dashboard cards

Changing the size of your cards on your dashboard helps highlight certain data views you will use often, or grabs other users' attention when viewing your dashboard for the first time.

To adjust a card's size, click and drag the bottom-right corner.

How to create and save a new custom dashboard

  1. From the left menu, click Dashboards and Reports.
  2. From the dashboard list, click the New Dashboard button. The New Dashboard modal appears.
  3. Enter a Dashboard Name and Description. The Description is optional, but it helps differentiate all of the dashboards at your organization.
  4. Click the Create Dashboard button.

Now that you’ve saved your new dashboard, it’s available in the dashboard list. To add data visualization cards to your dashboard, you’ve been directed to the Card Library. You can find, add, and customize cards for your dashboard. If you can’t find a card you want in the Card Library, build your own in the Card Builder.

See Dashboard Cards to learn how to customize your cards.

Get started with automation

Automation can greatly improve the efficiency of your team by tackling security issues as they emerge in your environment. Automatically containing threats, quickly notifying your team of suspicious activity, and tracking the progress of an investigation are just three of the ways that automation can help to streamline your security processes.

To take advantage of automation, you must install the Insight Orchestrator and configure automation connections. Once the Orchestrator is set up, you can explore our out-of-the-box workflow templates and use automation to enrich your investigations.

Automation core concepts

Review these automation core concepts to understand how to successfully apply automation across your detection, investigation, and response tasks.

  • Orchestrator: The Insight Orchestrator is an on-premise technological component that aids connection between your environment and the cloud.
  • Connections: Connections enable two-way communication between InsightIDR and third-party products, solutions, and tools. They are comprised of 2 parts: Plugins are the third-party integration sources with defined automation triggers and actions, while credentials are the sensitive secrets that enable authentication into a third-party source.
  • Workflows: Workflows are automated processes designed to quickly work through key tasks across detection, investigation, and response. They can be completely automated, or have human decision steps along the way so a member of your team can provide necessary context when it's needed most. Workflows can also be simple one-off actions, or a complex tree of orchestrated steps.
  • Triggers: Instead of requiring manual action, Triggers will automatically initiate a workflow in response to a security event. Triggers are currently only available for legacy detection rules.

How to set up automation

Step 1: Install and Activate the Insight Orchestrator

The Insight Orchestrator is a component installed on your network that gives InsightIDR the access it needs to automate security processes. You must have an orchestrator if you want to take advantage of the Automation features available in InsightIDR.

  1. From the InsightIDR left menu, click Data Collection.
  2. Click the Orchestrators tab, click Install Orchestrator, and follow the on-screen prompts. InsightConnect uses plugins to give you access to a wide variety of capabilities, from data manipulation to response actions. The Insight Orchestrator uses Docker containers to run individual plugins.

Docker container management

The Orchestrator uses Docker containers to run individual plugins. While you may log into the Orchestrator to view and manage these Docker containers, it is not necessary and is not advisable unless you have been directed to do so by the Rapid7 support team.

Step 2: Configure automation connections
InsightIDR includes several out-of-the-box Workflow Templates built for third-party tools that you can leverage for your security needs. In order to use these templates, you’ll need to set up Automation Connections. A connection comprises the credentials and required parameters, such as the application URL, that InsightIDR needs to access and authenticate to a third-party tool.

Task 1: Find Connection Requirements

InsightIDR currently supports automation for the following third-party integrations:

  • Active Directory
  • Cb Response
  • JIRA
  • Okta
  • ServiceNow

To review their connection requirements, see Configure Connections for Automation.

You can also view Connection requirements from the Extension Library or InsightConnect

  • In the Extension Library, search to find the plugin you want to connect, and click the Documentation tab. The Setup section shows the parameters required to configure a plugin connection.
  • In InsightConnect, navigate to Settings > Plugins & Tools, select the desired plugin, and click the Help tab. The Setup section shows the parameters required to configure a plugin connection.

Task 2: Create a Connection

After you gather information for your connections, you can configure the connections for your workflows. For instructions, see Configure Connections.

Task 3: Create a Credential

Most connections require a credential for use. Credential types include API keys, username/password pairs, and other sensitive information that is encrypted at rest and in transit. Credentials are saved separately from connections so that they may be reused across multiple connections as needed. Credentials used in your connections are encrypted on the Orchestrator itself. For more information about credential security, see Orchestrator Security.

  1. Click on the Choose a Credential input in the connection configuration menu and select Create New Credential.
  2. Enter a display name for your credential, then enter the credential itself.
  3. Click Save and proceed with your connection configuration.

For more information on editing credentials and using existing connections, see InsightConnect Create a Connection.

Step 3: Create a workflow from a template

Workflows are available for you to kick off when there is something in your investigation that requires you to take action. You can use out-of-the-box workflow templates to set up Workflows that do things like quarantine an asset, suspend a user account from within an investigation, and create JIRA tickets. These Workflows are based on Workflow Templates. These templates are workflows that do not have configured connections and are handy when you need to reuse Workflow steps with different connections.

Create your first workflow

After you’ve installed an Orchestrator, we recommend starting with the Open Source Alert Enrichment workflow to contextualize your investigations and alerts.

  1. Open the Investigation where you’d like to add automation and click Take Action.
  2. Click Enrichment Workflows from the Select an Action category.
  3. Click Enrich Alert Data with Open Source Plugins from the Select and Automation Action to Take dropdown.
  4. Follow the instructions to name your workflow, select connections, and configure details.
  5. Click Take Action to launch your workflow.

To configure other workflow templates, see Automation Workflow Templates.

Step 4: Use a workflow in an investigation

To begin using automation workflows to streamline your investigations, try an Automated Enrichment Workflow. InsightIDR offers built-in enrichment workflow templates that you can configure.

These workflows can process and enrich the following input data types:

  • Asset
  • Domain
  • IP Address
  • Process
  • URL
  • User

Most enrichment workflow templates are capable of processing four or more of these data types from the same investigation, but InsightIDR does not require all data types to be present in order for the workflow to run. As long as any enrichment workflow has viable input to process, it will return enriched content for your security teams to use.

Enrichment workflows rely on several third party and open source plugins to process your investigation data; ensure you have installed and activated an Insight Orchestrator.

Enable File Integrity Monitoring (FIM)

File Integrity Monitoring (FIM) allows you to audit changes to critical files and folders for compliance reasons on Windows systems running agent version 2.5.3.8 or later.

When you turn on FIM, the Insight Agent starts collecting FIM events. InsightIDR can then attribute users to file modification activity. You can create alerts based on certain file log events to notify you when one of your users modifies a critical file or folder.

How to configure FIM

Deploy the Network Sensor

During your first 45 days, you deployed Insight Agents and Collectors to monitor your assets for vulnerabilities and user behavior. While these components are responsible for collecting data on your assets, they do not account for network traffic, which is the data moving between your assets. To provide the network traffic visibility that you need for network traffic analysis (NTA), Rapid7 offers the Insight Network Sensor with multiple deployment options.

The Insight Network Sensor allows you to monitor, capture, and assess the end-to-end network traffic moving throughout your physical and virtual environment. InsightIDR can then leverage this network sensor data for network traffic analysis and IDS (Intrusion Detection System) detection rules.

Network Sensor deployment options

There are 3 options for deploying the network sensor. All options offer network traffic visibility, but are deployed and configured differently.

  • The Insight Network Sensor deployed on a physical server
  • The Insight Network Sensor deployed on a virtual machine
  • The Insight Network Sensor for AWS, which is deployed on an EC2 instance

To deploy an Insight Network Sensor, see the Insight Network Sensor Deployment Guide.

How IDS detection rules work

Intrusion Detection System (IDS) is an application that monitors for malicious activity and policy violations on your network. When configuring this event source in InsightIDR, the IDS data is attributed to the user and asset details page and allows you to search through the data. However, it does not produce alerts.

InsightIDR can collect events from these types of IDS/IPS devices:

  • Cisco FirePower
  • Corero IPS
  • Dell iSensor
  • F5 Networks BIG-IP Local Traffic Manager
  • HP TippingPoint
  • Juniper Junos
  • McAfee IDS
  • Metaflows IDS
  • Security Onion
  • Sentinel IPS
  • Snort IDS
  • Network Sensor

To collect IDS/IPS events, you need to configure the device to send syslog to the Collector. For a detailed list of the rules used to identify suspicious activity, see IDS detection rules powered by the Insight Network Sensor.

Tap into your Ultimate Exclusives: Enhanced Endpoint Telemetry, Enhanced Network Traffic Analysis, and Velociraptor

Discover Enhanced Endpoint Telemetry (EET)

While InsightIDR provides out-of-the-box detections for suspicious and malicious events, the information captured by the Insight Agent contains rich metadata that is useful for creating custom detections, accelerating alert investigations, and facilitating complete incident response.

With Enhanced Endpoint Telemetry (EET), you can:

  • Unlock all of the details of what happened on an endpoint before, after, and during an attack
  • Proactively hunt for threats or look for anomalous activity that may signal a bad actor at play
  • Create and tailor basic detection rules (formerly known as custom alerts) to align to your specific policies and security standards

To access EET data, you must have the latest version of the Insight Agent installed on your endpoints.

Add an EET card to a dashboard

To view your Enhanced Endpoint Telemetry data:

  • Navigate to one of your dashboards
  • Click Add Card
  • Select the Enhanced Endpoint Telemetry category from the list
  • Add one of the EET cards to your dashboard

Check out Enhanced Endpoint Telemetry for details on EET metadata, how to query your process start data, create alerts from custom queries.

Perform Enhanced Network Traffic Analysis (ENTA)

Enhanced Network Traffic Analysis (ENTA) gives you access to all raw network flow data and the rich metadata collected by the Network Traffic Sensor including IP addresses, ports, content based application recognition, and other metadata attributed to specific users and devices.

With ENTA, you can:

  • Create custom dashboards with network flow data to report on activity such as Top Users, Top Network Applications, Top Countries Connecting Inbound, Inbound Ports Active on Firewall, and many others. Check out the dashboard card library once you have deployed an Insight Network Sensor.
  • Accelerate investigations with more context and visibility, such as: network flow data to see what assets a client connected to, what network protocols were used, any inbound activity from external sources, and how much data was sent or received.
  • Accurately and reliably fingerprint the applications on your network with CBAR (Content-Based Application Recognition) using advanced deep packet inspection (DPI) techniques.
  • Build basic detection rules, hunt for other network threats, or drive forensic activities with network flow data. You can create basic detection rules to show when an external host is connected inbound using a network protocol such as RDP.
  • Root out weak protocol usage such as TLS1.1 using network flow records. The Insight Network Sensor will capture information such as encryption protocol and associated organization.

Add a pre-built NTA dashboard

To view your Enhanced Network Traffic Analysis data:

  • From the Dashboards and Reports page, open the Dashboard Library
  • Select the Network Traffic Analysis category from the list
  • Add one of the NTA dashboards
  • View your new NTA dashboard under All Dashboards

Add a network flow card to a dashboard

To view your network flow data:

  • Navigate your newly added NTA dashboard
  • Click Add Card
  • Select the one of the three Network Flow categories from the list
  • Add one of the network flow cards to your NTA dashboard

To take advantage of ENTA data, you must deploy the Insight Network Sensor.

Use Velociraptor for Digital Forensics and Incident Response (DFIR)

Velociraptor is an open source DFIR tool that provides a powerful and efficient way to hunt for specific artifacts and monitor activities across fleets of endpoints. As an InsightIDR Ultimate customer, you have access to Velociraptor integrated with InsightIDR and the Insight Platform, which adds DFIR capabilities to InsightIDR's investigate toolset for a greater level of monitoring and swifter responses to issues.

With integrated Velociraptor, you can:

To take advantage of Velociraptor, you must install the Insight Agent on your endpoints. Check out Velociraptor Integration to learn more about how Velociraptor works with InsightIDR and the Insight Platform.

Day 90+: Continue your InsightIDR Exploration and Education

Discovering new aspects of InsightIDR doesn't stop after your first 90 days! Rapid7 has workshops, monthly product updates, and in-product messages to keep you informed about new features and the value you can gain from them.

The Rapid7 Academy

The Rapid7 Academy holds trainings, webcasts, workshops, and more, all led by our Rapid7 experts.

  • On-demand training helps you get started with Rapid7 products, answer frequently asked questions, and recommend best practices.
  • Rapid7 Webcasts are hosted by Rapid7's product teams and provide a forum where customers can learn about best practices as well as what’s new in their Rapid7 products.
  • Virtual Instuctor-Led Training Courses are live training sessions broken down by product and available for enrollment.
  • Certification Exams are product-specific exams to help you demonstrate your knowledge of using Rapid7's solutions as a cybersecurity professional.
  • Product Workshops are Rapid7's free trainings on all things, all products, and average about an hour long.
Communications

To make sure you receive the Rapid7 communications that best suit your needs, set your communication preferences.

  • Whether it's an emergent cybersecurity threat, a product update, or a notice of service degradation for maintenance, we'll alert you with an in-product message to ensure you're aware of all that affects your environment.
  • Rapid7's research provides information on a variety of topics, such as, cloud misconfigurations, vulnerability management, detection and response, application security, and more.
  • Rapid7's blog offers conversational guidance and information from our security experts.
  • InsightIDR's Release Notes and in-product What's New guides are published at the end of each month to notify you of the product updates, improvements, and bug fixes that have occurred during the month.
Our Communities

Rapid7 supports a range of open-source projects. Consider joining one of our Open-Source communities!

  • AttackerKB captures, highlights, and expands on security researcher knowledge to shed light on the specific conditions and characteristics that make a vulnerability exploitable and useful to attackers.
  • Velociraptor provides you with the ability to more effectively respond to a wide range of digital forensic and cyber incident response investigations and data breaches.
  • Metasploit empowers and arms defenders to stay one step ahead of the game by verifying vulnerabilities, managing security assessments, and improving security awareness.
  • Recog is a framework for identifying products, services, operating systems, and hardware by matching fingerprints against data returned from various network probes.

Our customer advocacy program, Rapid7 Voice, provides you with a network of customers, offers the chance to deepen your security expertise, and provides the opportunity to share input on future product developments.

Rapid7 Support

When you run into any problems with InsightIDR, first search our help site for potential solutions. If you need additional assistance, please contact Rapid7 Support through the customer portal.