What are Data Sources?

An Data Source represents an asset that sends logs to the collector or the Insight Agent. This data is then converted to normalized JSON before it is ingested by InsightOps.

basic

Adding an Data Source

  1. Click the Add Data button at the top of any page.
  2. Make sure install the Insight Agent and/or the Collector in order to collect logs from your assets. Select Setup Agent or Setup Collectors from the top right menus.
  3. Select a data source to add from an app library, your system data, or your security data.

Log Collection

Below are the log collection methods InsightOps allows you to choose from when adding a data source.

Most Common

  • Listen for Syslog - Logs are being directed through a specific port on your network. Tell InsightOps at which port to "listen," and it will collect the logs.
  • Log Aggregator - Logs are currently being aggregated in a single place somewhere else, but can be forwarded to InsightOps

Other Methods

  • Tail File - InsightOps will watch a log file and ingest any new data that is added to it.
  • Watch Directory - InsightOps will watch a directory and ingest any new data that is added to it.
  • WMI - only used for Generic Windows Data Log in Raw Data.

Ports & Protocols

  • Port: A single log may be directed through a port on your network; configure InsightOps to collect data from that port
  • UDP Protocol: a connectionless protocol that sends individual packets of data over your network. Referred to as the "best effort" protocol, although there is no guarantee of data delivery.
  • TCP Protocol: a protocol that sends data in a byte stream from computer to computer and guarantees intact data in the same order it was sent in.

Please see Token TCP, HTTP POST, and Plain TCP/UDP for additional input information.

How InsightOps Stores Data

InsightOps stores data via Amazon Web Services (AWS) which you can configure through S3 Archiving.

Data Collected by InsightOps

The Collector Data sources are data streams that generate log events and ingested by the Collector.

InsightOps monitors the following set of fields from Data source logs:

Active Directory

  • Timestamp
  • Action
  • Source User
  • Source Account
  • Target User
  • Target Account
  • Group
  • Group Scope
  • Group Domain

Asset Authentication

  • Timestamp
  • Source Asset
  • Destination Asset
  • Source Asset Address
  • Destination Asset Address
  • Destination User
  • Destination Account
  • Destination Domain
  • Destination Account SID
  • Login Type
  • Result
  • New Authentication
  • New Source Authentication
  • New Source for Account
  • Service

Cloud Service Administrator Activity

  • Timestamp
  • Service
  • Action
  • Source User
  • Source Account
  • Target User
  • Target Account

DNS

  • Timestamp
  • Asset
  • User
  • Source Address
  • Query
  • Public Suffix
  • Top Private Domain

File Access Activity

  • Timestamp
  • User
  • Account
  • Account Domain
  • Source Address
  • Service
  • Target Address
  • File Path
  • File Name
  • File Extension
  • File Share
  • Access Type

Firewall Activity

  • Timestamp
  • Asset
  • User
  • Source Address
  • Source Port
  • Destination Address
  • Destination Port
  • Connection Status
  • Direction
  • GEOIP Organization
  • GEOIP Country Code
  • GEOIP Country Name
  • GEOIP City
  • GEOIP Region

Host to IP Observations

  • Timestamp
  • Action
  • HostID
  • IP
  • Observation Status

IDS Alerts

  • Timestamp
  • Asset
  • User
  • Signature
  • Source IP
  • Destination
  • Description
  • Severity
  • Protocol
  • Generator ID
  • Source Port
  • Destination Port

Ingress Authentication (OWA/ActiveSync)

  • Timestamp
  • User
  • Account
  • Result
  • Source IP
  • Service
  • GEOIP Organization
  • GEOIP Country Code
  • GEOIP Country Name
  • GEOIP City
  • GEOIP Region

Raw Logs (Generic Syslog and Windows Event Log)

  • Timestamp
  • Host Name
  • Event Code
  • Description
  • Package Name
  • Target User Name
  • Workstation
  • Status

SSO Authentication

  • Timestamp
  • User
  • Account
  • Source IP
  • Service
  • SSO Provider
  • GEOIP Organization
  • GEOIP Country Code
  • GEOIP Country Name
  • GEOIP City
  • GEOIP Region