Agent-based policies - Bulk Export API

Introduction

Using Bulk Export API, you can bulk export your Agent-Based Policy data using a GraphQL API. The data is returned in Parquet format and can be downloaded for use in your business intelligence tools.

To bulk export API data to Parquet files, complete the following:

  1. Choose the endpoint for your region.
  2. Initiate the generation of export files.
  3. Query the ID returned.
  4. Download the parquet files.

Authorization

Permissions

You require Platform Administrator permissions to carry out API Bulk Export.

Authorization is performed by passing your API user key via a single HTTP header: X-Api-Key. The user key must be passed to all requests. This can be generated from the Insight Platform key management page. For information on generating and managing API keys, see Managing Platform API Keys.

Endpoints

RegionURI
United States - 1https://us.api.insight.rapid7.com/export/graphql
United States - 2https://us2.api.insight.rapid7.com/export/graphql
United States - 3https://us3.api.insight.rapid7.com/export/graphql
Europehttps://eu.api.insight.rapid7.com/export/graphql
Canadahttps://ca.api.insight.rapid7.com/export/graphql
Australiahttps://au.api.insight.rapid7.com/export/graphql
Japanhttps://ap.api.insight.rapid7.com/export/graphql

Initiating the export

Complete these steps to create and retrieve the export files in Parquet.

Step 1: Mutation to initiate export

The mutations initiate the generation of the export files for Policy and Vulnerability data, respectively:

Policy:

 
1
mutation CreatePolicyExport { 
2
    createPolicyExport(input:{}) {
3
        id
4
    }
5
}

Step 2: Export query

To retrieve the URLs to download the Parquet files, you must query the ID returned when initiating the export. The results will return a URL where you can download the Parquet files.

Export considerations

The data is refreshed by the system once a day. You can make multiple export requests, however overuse of this method may result in throttling.

 
1
query GetExport {
2
    export(id: "YzY1ODk5YzQtNjkwNi00MDRjLTk3NDQtNjRhOGNkNWFkNDIx"){
3
        id
4
        status
5
        dataset
6
        timestamp
7
        result {
8
             prefix
9
             urls
10
       }
11
    }
12
}
13

Parquet files

These schemas provide an overview of the fields returned in the files at the URLs provided:

Agent-Based Policy Export

asset

FieldTypeDefinitionExample
orgIdStringOrganization IDa08de390-bb6a-4297-b1df-9ee58c7beb7a
assetIdStringAsset ID2b2b6a57-9136-4874-8af1-3f9f0a6de60a-default-asset-1
agentIdStringAgent ID586d68c92af55c27b7bfecf7f6df0cb4
awsInstanceIdStringThe Amazon Web Services instance ID of the asset, if applicablei-0e1cc483957bc29d8
azureResourceIdStringThe Azure resource identifier of the asset, if applicable
gcpObjectIdStringThe Google Cloud Platform identifier of the asset, if applicable
macStringThe primary MAC address of the asset0050568A103C
ipStringThe primary IP address of the asset0.0.0.0
hostNameStringThe primary hostname of the assettesthost.us
osArchitectureStringArchitecture of the OS on the assetx86_64
osFamilyStringFamily of the OS on the assetWindows
osProductStringProduct of the OS on the assetWindows Server 2016 Standard Edition
osVendorStringVendor of the OS on the assetMicrosoft
osVersionStringThe OS version on the asset1607
osTypeStringType of OS on the assetServer
sitesListArray of sites the asset belongs to[site1, site2]

asset_policy

FieldTypeDefinitionExample
orgId String Organization ID

a08de390-bb6a-4297-b1df-9ee58c7beb7a

assetId String Asset ID

2b2b6a57-9136-4874-8af1-3f9f0a6de60a-default-asset-1

benchmarkNaturalId String The natural ID of the XCCDF benchmark

xccdf_org.cisecurity.benchmarks_benchmark_2.0.0_CIS_Google_Chrome_Benchmark

profileNaturalId String The natural profile of the XCCDF benchmark

xccdf_org.cisecurity.benchmarks_profile_Level_1_L1_-_CorporateEnterprise_Environment_general_use

benchmarkVersion String The version of the XCCDF benchmark

2.0.0

ruleNaturalId String The natural ID of the XCCDF rule

xccdf_org.cisecurity.benchmarks_rule_3.6_L1_Ensure_Control_how_Chrome_Cleanup_reports_data_to_Google_is_set_to_Disabled

ruleTitle String The title of the XCCDF rule

3.6. (L1) Ensure 'Control how Chrome Cleanup reports data to Google' is set to 'Disabled'

finalStatus String The final status of the rule finding, after applying overrides

FAIL

proof String Text describing how the status was determined

<p><p>Based on the following 1 results:<ol><li><p><ol><li><p>At least one specified Windows registry information entry must match the given criteria. At least one evaluation must pass.<Table TableTitle=""><tr RowTitle=""><td>The specified Windows registry information entry was not found based on the given criteria:</td></tr><tr RowTitle=""><td>hive: HKEY_LOCAL_MACHINE</td></tr><tr RowTitle=""><td>key: SOFTWARE\Policies\Google\Chrome</td></tr><tr RowTitle=""><td>name: ChromeCleanupReportingEnabled</td></tr></Table></p></li></ol></p></li></ol></p></p>

lastAssessmentTimestamp The last time the policy was assessed

2022-12-06T04:07:44.471Z

benchmarkTitle String The title of the XCCDF benchmark

CIS Google Chrome Benchmark

profileTitle String The title of the XCCDF profile

Level 1 (L1) - Corporate/Enterprise Environment (general use)

publisher String The publisher of the policy

CIS

fixTexts List Data describing how to bring a target system into compliance with the rule

fixTexts=[ <xhtml:div xmlns="http://checklists.nist.gov/xccdf/1.2" xmlns:cc6="http://cisecurity.org/20-cc/v6.1" xmlns:cc7="http://cisecurity.org/20-cc/v7.0" xmlns:notes="http://benchmarks.cisecurity.org/notes" xmlns:xccdf="http://checklists.nist.gov/xccdf/1.2" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <xhtml:p> <xhtml:p> To establish the recommended configuration via Group Policy, set the following UI path to <xhtml:span class="inline_block">Disabled</xhtml:span> : </xhtml:p> <xhtml:code class="code_block">Computer Configuration\Administrative Templates\Google\Google Chrome\Control how Chrome Cleanup reports data to Google </xhtml:code> <xhtml:p class="bold">Impact:</xhtml:p> <xhtml:p> < xhtml:p> Chrome Cleanup detected unwanted software, will no longer report metadata about the scan to Google.</xhtml:p> </xhtml:div> ]

rationales List Descriptive text giving rationale or motivations for abiding by the rule

[<xhtml:p xmlns="http://checklists.nist.gov/xccdf/1.2" xmlns:cc6="http://cisecurity.org/20-cc/v6.1" xmlns:cc7="http://cisecurity.org/20-cc/v7.0" xmlns:notes="http://benchmarks.cisecurity.org/notes" xmlns:xccdf="http://checklists.nist.gov/xccdf/1.2" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">Anonymous crash/usage data can be used to identify people, companies and information, which can be considered data ex-filtration from company systems.</xhtml:p>]

Downloading the Parquet files

  • Each time URLs are generated to download the files, the URLs are valid for 15 minutes. The files can be downloaded as many times as needed within the 15 minute time frame.
  • These files are retained for 30 days only. Attempting to query the exports after 30 days will return an error message.