Data Collection Overview

Data collection is the process that allows you to send you log data to InsightOps from applications and assets, whether they're hosted or on-premises, physical or virtual.

Additionally, data collection reduces the need for storing lots of log data locally. Multiple logs can be centralized and searched in one place even, if they originate from disparate locations. Once properly configured, data collection also allows InsightOps to send you alerts and analyze your data.

How Does InsightOps Collect Data?

InsightOps collects data from your assets (endpoints) and your network in five distinct ways: the Insight Agent, the Collector, REST API, syslog, and application logging.

The Insight Agent

The Insight Agent is installed on individual assets and sends asset information to InsightOps. It also sends hardware metrics and usage details (disk usage, etc), and sends log files that the user specifies - typically system or application logs. And, if running on a Windows host, the agent will send log detail from the windows event log.

Please ensure that your assets meet the technical requirements outlined here. You can also read more about the Insight Agent here.

Syslog

Logging from Systems allows you to directly send log data to InsightOps. Learn more about syslog here.

Application Logging

If you are programming an application, you can use one of the many language libraries to send data to InsightOps.

The Collector

The Collector ingests information about your network via connected Data Sources. You can send your logs to InsightOps directly via the Insight Agent, or you can use the Collector to forward your logs.

Using the Collector will normalize your data, or transform log data from multiple diverse sources into a common JSON format and extract standard information like hostnames, timestamps, error levels, etc. Normalization allows you to run more advanced queries on your endpoint logs and enhance your data visualization.

Please ensure that your network meets the technical requirements outlined here. You can also read more about the Collector here.