Agent-based policies - Bulk Export API

Introduction

Using Bulk Export API, you can bulk export your Agent-Based Policy data using a GraphQL API. The data is returned in Parquet format and can be downloaded for use in your business intelligence tools.

To bulk export API data to Parquet files, complete the following:

Choose the endpoint for your region.
Initiate the generation of export files.
Query the ID returned.
Download the parquet files.

Authorization

⚠️

Permissions

You need an Organization API key or a user API key with Platform Administrator permissions to use the Bulk Export API.

Authorization is performed by passing your API key using the X-Api-Key header parameter. The API key must be passed with all requests. To learn how to generate and manage API keys, see Managing Platform API Keys .

Endpoints

Region	URI
United States - 1	https://us.api.insight.rapid7.com/export/graphql
United States - 2	https://us2.api.insight.rapid7.com/export/graphql
United States - 3	https://us3.api.insight.rapid7.com/export/graphql
Europe	https://eu.api.insight.rapid7.com/export/graphql
Canada	https://ca.api.insight.rapid7.com/export/graphql
Australia	https://au.api.insight.rapid7.com/export/graphql
Japan	https://ap.api.insight.rapid7.com/export/graphql

Initiating the export

Complete these steps to create and retrieve the export files in Parquet.

Step 1: Mutation to initiate export

The mutations initiate the generation of the export files for Policy and Vulnerability data, respectively:

Policy:


mutation CreatePolicyExport { 
    createPolicyExport(input:{}) {
        id
    }
}

Step 2: Export query

To retrieve the URLs to download the Parquet files, you must query the ID returned when initiating the export. The results will return a URL where you can download the Parquet files.

⚠️

Export considerations

The data is refreshed by the system once a day. You can make multiple export requests, however overuse of this method may result in throttling.


query GetExport {
    export(id: "YzY1ODk5YzQtNjkwNi00MDRjLTk3NDQtNjRhOGNkNWFkNDIx"){
        id
        status
        dataset
        timestamp
        result {
             prefix
             urls
       }
    }
}

Parquet files

These schemas provide an overview of the fields returned in the files at the URLs provided:

Agent-Based Policy Export

asset

Field	Type	Definition	Example
orgId	String	Organization ID	`a08de390-bb6a-4297-b1df-9ee58c7beb7a`
assetId	String	Asset ID	`2b2b6a57-9136-4874-8af1-3f9f0a6de60a-default-asset-1`
agentId	String	Agent ID	`586d68c92af55c27b7bfecf7f6df0cb4`
awsInstanceId	String	The Amazon Web Services instance ID of the asset, if applicable	`i-0e1cc483957bc29d8`
azureResourceId	String	The Azure resource identifier of the asset, if applicable
gcpObjectId	String	The Google Cloud Platform identifier of the asset, if applicable
mac	String	The primary MAC address of the asset	`0050568A103C`
ip	String	The primary IP address of the asset	`0.0.0.0`
hostName	String	The primary hostname of the asset	`testhost.us`
osArchitecture	String	Architecture of the OS on the asset	`x86_64`
osFamily	String	Family of the OS on the asset	`Windows`
osProduct	String	Product of the OS on the asset	`Windows Server 2016 Standard Edition`
osVendor	String	Vendor of the OS on the asset	`Microsoft`
osVersion	String	The OS version on the asset	`1607`
osType	String	Type of OS on the asset	`Server`
osDescription	String	The description for the operating system	`Microsoft Windows Server 2016 Standard Edition 1607`
riskScore	Double	The asset risk score	`10000.0`
sites	List	Array of sites the asset belongs to	`[site1, site2]`
assetGroups	List	The groups the asset is part of	`[group1, group2]`
tags	List	The tags present on the asset	`[{name: tag1, tagType: Owner}, {name: tag2, tagType: Location}]`

asset_policy

Field	Type	Definition	Example
orgId	String	Organization ID	`a08de390-bb6a-4297-b1df-9ee58c7beb7a`
assetId	String	Asset ID	`2b2b6a57-9136-4874-8af1-3f9f0a6de60a-default-asset-1`
benchmarkNaturalId	String	The natural ID of the XCCDF benchmark	`xccdf_org.cisecurity.benchmarks_benchmark_2.0.0_CIS_Google_Chrome_Benchmark`
profileNaturalId	String	The natural profile of the XCCDF benchmark	`xccdf_org.cisecurity.benchmarks_profile_Level_1_L1_-_CorporateEnterprise_Environment_general_use`
benchmarkVersion	String	The version of the XCCDF benchmark	`2.0.0`
ruleNaturalId	String	The natural ID of the XCCDF rule	`xccdf_org.cisecurity.benchmarks_rule_3.6_L1_Ensure_Control_how_Chrome_Cleanup_reports_data_to_Google_is_set_to_Disabled`
ruleTitle	String	The title of the XCCDF rule	`3.6. (L1) Ensure 'Control how Chrome Cleanup reports data to Google' is set to 'Disabled'`
finalStatus	String	The final status of the rule finding, after applying overrides	`FAIL`
proof	String	Text describing how the status was determined	<p><p>Based on the following 1 results:<ol><li><p><ol><li><p>At least one specified Windows registry information entry must match the given criteria. At least one evaluation must pass.<Table TableTitle=""><Table.Tr RowTitle=""><Table.Td>The specified Windows registry information entry was not found based on the given criteria:</Table.Td></Table.Tr><Table.Tr RowTitle=""><Table.Td>hive: HKEY_LOCAL_MACHINE</Table.Td></Table.Tr><Table.Tr RowTitle=""><Table.Td>key: SOFTWARE\Policies\Google\Chrome</Table.Td></Table.Tr><Table.Tr RowTitle=""><Table.Td>name: ChromeCleanupReportingEnabled</Table.Td></Table.Tr></Table></p></li></ol></p></li></ol></p></p>
lastAssessmentTimestamp	Timestamp	The last time the policy was assessed	`2022-12-06T04:07:44.471Z`
benchmarkTitle	String	The title of the XCCDF benchmark	`CIS Google Chrome Benchmark`
profileTitle	String	The title of the XCCDF profile	`Level 1 (L1) - Corporate/Enterprise Environment (general use)`
publisher	String	The publisher of the policy	`CIS`
fixTexts	List	Data describing how to bring a target system into compliance with the rule	fixTexts=[ <xhtml:div xmlns="http://checklists.nist.gov/xccdf/1.2" xmlns:cc6="http://cisecurity.org/20-cc/v6.1" xmlns:cc7="http://cisecurity.org/20-cc/v7.0" xmlns:notes="http://benchmarks.cisecurity.org/notes" xmlns:xccdf="http://checklists.nist.gov/xccdf/1.2" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <xhtml:p> <xhtml:p> To establish the recommended configuration via Group Policy, set the following UI path to <xhtml:span class="inline_block">Disabled</xhtml:span> : </xhtml:p> <xhtml:code class="code_block">Computer Configuration\Administrative Templates\Google\Google Chrome\Control how Chrome Cleanup reports data to Google </xhtml:code> <xhtml:p class="bold">Impact:</xhtml:p> <xhtml:p> < xhtml:p> Chrome Cleanup detected unwanted software, will no longer report metadata about the scan to Google.</xhtml:p> </xhtml:div> ]
rationales	List	Descriptive text giving rationale or motivations for abiding by the rule	[<xhtml:p xmlns="http://checklists.nist.gov/xccdf/1.2" xmlns:cc6="http://cisecurity.org/20-cc/v6.1" xmlns:cc7="http://cisecurity.org/20-cc/v7.0" xmlns:notes="http://benchmarks.cisecurity.org/notes" xmlns:xccdf="http://checklists.nist.gov/xccdf/1.2" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">Anonymous crash/usage data can be used to identify people, companies and information, which can be considered data ex-filtration from company systems.</xhtml:p>]

Downloading the Parquet files

Each time URLs are generated to download the files, the URLs are valid for 15 minutes. The files can be downloaded as many times as needed within the 15 minute time frame.
These files are retained for 30 days only. Attempting to query the exports after 30 days will return an error message.