Data Collection Overview
Data collection is the process that allows you to send you log data to InsightOps from applications and assets, whether they're hosted or on-premises, physical or virtual.
Additionally, data collection reduces the need for storing lots of log data locally. Multiple logs can be centralized and searched in one place even, if they originate from disparate locations. Once properly configured, data collection also allows InsightOps to send you alerts and analyze your data.
How Does InsightOps Collect Data?
InsightOps collects data from your assets (endpoints) and your network in five distinct ways: the Insight Agent, the Collector, REST API, syslog, and application logging.
The Insight Agent
The Insight Agent is installed on individual assets and sends asset information to InsightOps. It also sends hardware metrics and usage details (disk usage, etc), and sends log files that the user specifies - typically system or application logs. And, if running on a Windows host, the agent will send log detail from the windows event log.
Logging from Systems allows you to directly send log data to InsightOps. Learn more about syslog here.
If you are programming an application, you can use one of the many language libraries to send data to InsightOps.
The Collector ingests information about your network via connected Data Sources. You can send your logs to InsightOps directly via the Insight Agent, or you can use the Collector to forward your logs.
Using the Collector will normalize your data, or transform log data from multiple diverse sources into a common JSON format and extract standard information like hostnames, timestamps, error levels, etc. Normalization allows you to run more advanced queries on your endpoint logs and enhance your data visualization.