Bulk export agent-based policies and vulnerabilities with the API
Overview
Using the Bulk Export API, you can bulk export your agent-based policies, asset, vulnerabilities and asset, or vulnerabilty remediation data using a GraphQL API. The data is returned in Parquet format and can be downloaded for use in your business intelligence tools.
To bulk export API data to Parquet files, complete the following tasks:
- Gather the information required to make a request
- Make a request
- Check the status of the export
- Download the files
Task 1: Gather required information
To sucessfully run a Bulk Export request, you will need multiple pieces of information:
- An API key to authorize the request
- The relevant URI for your region
- The mutation for the data you want to export
As you complete task 1, copy the information you get somewhere safe for use in task 2.
Gain authorization
Permissions
You need an Organization API key or a user API key with Platform Administrator permissions to use the Bulk Export API.
Authorization is performed by passing your API key using the X-Api-Key header parameter. The API key must be passed with all requests. To learn how to generate and manage API keys, see Managing Platform API Keys .
Select your region
Select the URI that matches the region associated with the API key you obtained for authorization.
| Region | URI |
|---|---|
| United States - 1 | https://us.api.insight.rapid7.com/export/graphql |
| United States - 2 | https://us2.api.insight.rapid7.com/export/graphql |
| United States - 3 | https://us3.api.insight.rapid7.com/export/graphql |
| Europe | https://eu.api.insight.rapid7.com/export/graphql |
| Canada | https://ca.api.insight.rapid7.com/export/graphql |
| Australia | https://au.api.insight.rapid7.com/export/graphql |
| Japan | https://ap.api.insight.rapid7.com/export/graphql |
Select export type
Select a mutation based on the data you require:
Agent-based policy and all assets
mutation CreatePolicyExport {
createPolicyExport(input:{}) {
id
}
}All vulnerabilities and all assets
mutation CreateVulnerabilityExport {
createVulnerabilityExport(input:{}) {
id
}
}Vulnerability remediations
mutation CreateVulnerabilityRemediationExport($input: VulnerabilityRemediationExportConfiguration!) {
createVulnerabilityRemediationExport(input: $input)
{
id
}
}Additionally for this request, you must specify a date range, replacing startDate and endDate with date ranges from August 2025 onwards. startDate and endDate must not be the same date.
Date limitations
For this additional variable, startDate and endDate:
- Must not be the same date
- Must not be further than 31 days apart
{
"input": {
"startDate": "2025-08-03",
"endDate": "2025-08-24"
}
}Task 2: Make a request
This guide shows how to complete this procedure using Postman.
You can use any GraphQL API client, but the steps outlined here are specific to Postman. All information from task 1 will be the same regardless of the tool you use.
Now that you have the required information from task 1, you can make a request to retrieve the URLs to download Parquet files.
Export considerations
The data is refreshed by the system once a day. You can make multiple export requests, however overuse of this method may result in throttling.
To make a request:
- In Postman, click the three dots beside the relevant Collection and select Add Request.
- Change the query to POST using the dropdown in the query bar.
- Paste the region URI from task 1.
- Go to the Headers tab.
- Paste
X-Api-Keyinto the Key column. - Paste the API key from task 1 into the Value column.
- Paste
- Go to the Body tab.
- Select GraphQL from the options at the top of the tab.
- In the Query section, paste the mutation you selected in task 1.
- Click Send in the query bar. This will begin to run the request.
- Under Body you will see a response, where
EXAMPLEIDwill be a unique string:
{
"data": {
"createvulnerabilityExport": {
"id" : "EXAMPLEID"
}
}
}Copy this ID and keep it somewhere safe for use in task 3.
Task 3: Check the status of the export
Once you have run the initial request in task 2, you will need to create an additional request to check the status of the export. This allows you to verify that the initial request ran successfully and to access the URLs you need to download to export the data.
To check the export status:
- In Postman, click the three dots beside the relevant Collection and select Add Request.
- Change the query to GET using the dropdown in the query bar.
- Paste the region URI from task 1.
- Go to the Headers tab.
- Paste
X-Api-Keyinto the Key column. - Paste the API key from task 1 into the Value column.
- Paste
- Go to the Body tab.
- Select GraphQL from the options at the top of the tab.
- In the Query section, paste this query, replacing
EXAMPLEIDwith the unique string you copied in task 2.
query GetExport {
export(id: "EXAMPLEID"){
id
status
dataset
timestamp
result {
prefix
urls
}
}
}- Click Send in the query bar. This will begin to run the request.
- Under Body you will see a response which will outline the status of the initial request from task 2.
Task 4: Download the Parquet files
Once the status of the request from task 3 becomes SUCCEEDED, you will see mutliple URLs listed, each with a prefix value.
Each prefix will outline what data is included in that URL, for example the URL tied to "prefix": "asset" will be an export of asset data.
You may have more than one URL relating to one prefix. This is dependant on the size of your environment and the amount of information contained in the export.
- Each time download URLs are generated, they remain valid for 15 minutes. Within this window, files can be downloaded as many times as needed.
- These URLs can be regenerated at any time within 30 days of the initial request.
- Exported files are retained for 30 days. After that, attempts to access the files will result in an error.
Parquet files
These schemas provide an overview of the fields returned in the files at the URLs provided:
Agent-based policy export
asset
| Field | Type | Definition | Example |
|---|---|---|---|
| orgId | String | Organization ID | a08de390-bb6a-4297-b1df-9ee58c7beb7a |
| assetId | String | Asset ID | 2b2b6a57-9136-4874-8af1-3f9f0a6de60a-default-asset-1 |
| agentId | String | Agent ID | 586d68c92af55c27b7bfecf7f6df0cb4 |
| awsInstanceId | String | The Amazon Web Services instance ID of the asset, if applicable | i-0e1cc483957bc29d8 |
| azureResourceId | String | The Azure resource identifier of the asset, if applicable | |
| gcpObjectId | String | The Google Cloud Platform identifier of the asset, if applicable | |
| mac | String | The primary MAC address of the asset | 0050568A103C |
| ip | String | The primary IP address of the asset | 0.0.0.0 |
| hostName | String | The primary hostname of the asset | testhost.us |
| osArchitecture | String | Architecture of the OS on the asset | x86_64 |
| osFamily | String | Family of the OS on the asset | Windows |
| osProduct | String | Product of the OS on the asset | Windows Server 2016 Standard Edition |
| osVendor | String | Vendor of the OS on the asset | Microsoft |
| osVersion | String | The OS version on the asset | 1607 |
| osType | String | Type of OS on the asset | Server |
| osDescription | String | The description for the operating system | Microsoft Windows Server 2016 Standard Edition 1607 |
| riskScore | Double | The asset risk score | 10000.0 |
| sites | List | Array of sites the asset belongs to | [site1, site2] |
| assetGroups | List | The groups the asset is part of | [group1, group2] |
| tags | List | The tags present on the asset | [{name: tag1, tagType: Owner}, {name: tag2, tagType: Location}] |
asset_policy
| Field | Type | Definition | Example |
|---|---|---|---|
| orgId | String | Organization ID | a08de390-bb6a-4297-b1df-9ee58c7beb7a |
| assetId | String | Asset ID | 2b2b6a57-9136-4874-8af1-3f9f0a6de60a-default-asset-1 |
| benchmarkNaturalId | String | The natural ID of the XCCDF benchmark | xccdf_org.cisecurity.benchmarks_benchmark_2.0.0_CIS_Google_Chrome_Benchmark |
| profileNaturalId | String | The natural profile of the XCCDF benchmark | xccdf_org.cisecurity.benchmarks_profile_Level_1_L1_-_CorporateEnterprise_Environment_general_use |
| benchmarkVersion | String | The version of the XCCDF benchmark | 2.0.0 |
| ruleNaturalId | String | The natural ID of the XCCDF rule | xccdf_org.cisecurity.benchmarks_rule_3.6_L1_Ensure_Control_how_Chrome_Cleanup_reports_data_to_Google_is_set_to_Disabled |
| ruleTitle | String | The title of the XCCDF rule | 3.6. (L1) Ensure 'Control how Chrome Cleanup reports data to Google' is set to 'Disabled' |
| finalStatus | String | The final status of the rule finding, after applying overrides | FAIL |
| proof | String | Text describing how the status was determined | <p><p>Based on the following 1 results:<ol><li><p><ol><li><p>At least one specified Windows registry information entry must match the given criteria. At least one evaluation must pass.<Table TableTitle=""><Table.Tr RowTitle=""><Table.Td>The specified Windows registry information entry was not found based on the given criteria:</Table.Td></Table.Tr><Table.Tr RowTitle=""><Table.Td>hive: HKEY_LOCAL_MACHINE</Table.Td></Table.Tr><Table.Tr RowTitle=""><Table.Td>key: SOFTWARE\Policies\Google\Chrome</Table.Td></Table.Tr><Table.Tr RowTitle=""><Table.Td>name: ChromeCleanupReportingEnabled</Table.Td></Table.Tr></Table></p></li></ol></p></li></ol></p></p> |
| lastAssessmentTimestamp | Timestamp | The last time the policy was assessed | 2022-12-06T04:07:44.471Z |
| benchmarkTitle | String | The title of the XCCDF benchmark | CIS Google Chrome Benchmark |
| profileTitle | String | The title of the XCCDF profile | Level 1 (L1) - Corporate/Enterprise Environment (general use) |
| publisher | String | The publisher of the policy | CIS |
| fixTexts | List | Data describing how to bring a target system into compliance with the rule | fixTexts=[ <xhtml:div xmlns="http://checklists.nist.gov/xccdf/1.2" xmlns:cc6="http://cisecurity.org/20-cc/v6.1" xmlns:cc7="http://cisecurity.org/20-cc/v7.0" xmlns:notes="http://benchmarks.cisecurity.org/notes" xmlns:xccdf="http://checklists.nist.gov/xccdf/1.2" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><xhtml:p><xhtml:p>To establish the recommended configuration via Group Policy, set the following UI path to <xhtml:span class="inline_block">Disabled</xhtml:span> : </xhtml:p> <xhtml:code class="code_block">Computer Configuration\Administrative Templates\Google\Google Chrome\Control how Chrome Cleanup reports data to Google </xhtml:code> <xhtml:p class="bold">Impact:</xhtml:p> <xhtml:p> < xhtml:p> Chrome Cleanup detected unwanted software, will no longer report metadata about the scan to Google.</xhtml:p> </xhtml:div> ] |
| rationales | List | Descriptive text giving rationale or motivations for abiding by the rule | [<xhtml:p xmlns="http://checklists.nist.gov/xccdf/1.2" xmlns:cc6="http://cisecurity.org/20-cc/v6.1" xmlns:cc7="http://cisecurity.org/20-cc/v7.0" xmlns:notes="http://benchmarks.cisecurity.org/notes" xmlns:xccdf="http://checklists.nist.gov/xccdf/1.2" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">Anonymous crash/usage data can be used to identify people, companies and information, which can be considered data ex-filtration from company systems.</xhtml:p>] |
Vulnerability export
asset
| Field | Type | Definition | Example |
|---|---|---|---|
| orgId | String | Organization ID | a08de390-bb6a-4297-b1df-9ee58c7beb7a |
| assetId | String | Asset ID | 2b2b6a57-9136-4874-8af1-3f9f0a6de60a-default-asset-1 |
| agentId | String | Agent ID | 586d68c92af55c27b7bfecf7f6df0cb4 |
| awsInstanceId | String | The Amazon Web Services instance ID of the asset, if applicable | i-0e1cc483957bc29d8 |
| azureResourceId | String | The Azure resource identifier of the asset, if applicable | |
| gcpObjectId | String | The Google Cloud Platform identifier of the asset, if applicable | |
| mac | String | The primary MAC address of the asset | 0050568A103C |
| ip | String | The primary IP address of the asset | 0.0.0.0 |
| hostName | String | The primary hostname of the asset | testhost.us |
| osArchitecture | String | Architecture of the OS on the asset | x86_64 |
| osFamily | String | Family of the OS on the asset | Windows |
| osProduct | String | Product of the OS on the asset | Windows Server 2016 Standard Edition |
| osVendor | String | Vendor of the OS on the asset | Microsoft |
| osVersion | String | The OS version on the asset | 1607 |
| osType | String | Type of OS on the asset | Server |
| osDescription | String | The description for the operating system | Microsoft Windows Server 2016 Standard Edition 1607 |
| riskScore | Double | The asset risk score | 10000.0 |
| sites | List | Array of sites the asset belongs to | [site1, site2] |
| assetGroups | List | The groups the asset is part of | [group1, group2] |
| tags | List | The tags present on the asset | [{name: tag1, tagType: Owner}, {name: tag2, tagType: Location}] |
asset_vulnerability
| Field | Type | Definition | Example |
|---|---|---|---|
| orgId | String | Organization ID | a08de390-bb6a-4297-b1df-9ee58c7beb7a |
| assetId | String | Asset ID | 2b2b6a57-9136-4874-8af1-3f9f0a6de60a-default-asset-1 |
| vulnId | String | Vulnerability ID | msft-cve-2023-29372 |
| port | Integer | The scanned port the vulnerability is present on, if applicable | 22 |
| protocol | String | The scanned protocol the vulnerability is present on, if applicable | TCP |
| nic | String | The scanned network interface, if applicable | |
| proof | String | Proof describing how the asset is vulnerable to the vulnerability | <p><p>Vulnerable OS: Microsoft Windows Server 2012 R2 Standard Edition<p></p></p><p>Based on the following 3 results:<ol><li><p>Microsoft patch KB5027282 is not installed.</p></li><li><p>Microsoft patch KB5027271 is not installed.</p></li><li><p><ul><li>HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion<ul><li>UBR - contains unexpected value 20337</li></ul></li></ul></p></li></ol></p></p> |
| firstFoundTimestamp | Timestamp | Timestamp of when the vulnerability was found on the asset | 2023-11-06T11:40:37.000Z |
| title | String | The title of the vulnerability | Microsoft CVE-2023-29372: Microsoft WDAC OLE DB provider for SQL Server Remote Code Execution Vulnerability |
| description | String | The description of the vulnerability, containing a mix of HTML/XML content | Microsoft CVE-2023-29372: Microsoft WDAC OLE DB provider for SQL Server Remote Code Execution Vulnerability |
| cvssAccessComplexity | String | The CVSS V2 access complexity of the vulnerability | M |
| cvssAccessVector | String | The CVSS V2 access vector of the vulnerability | N |
| cvssAuthentication | String | The CVSS V2 authentication of the vulnerability | N |
| cvssAvailabilityImpact | String | The CVSS V2 availability impact of the vulnerability | C |
| cvssConfidentialityImpact | String | The CVSS V2 confidentiality impact of the vulnerability | C |
| cvssIntegrityImpact | String | The CVSS V2 integrity impact of the vulnerability | C |
| cvssScore | Double | The CVSS V2 score of the vulnerability | 8.8 |
| cvssV3AttackVector | String | The CVSS V3 attack vector (AV) of the vulnerability | Network |
| cvssV3AttackComplexity | String | The CVSS V3 attack complexity (AC) of the vulnerability | Low |
| cvssV3PrivilegesRequired | String | The CVSS V3 privileges required (PR) of the vulnerability | None |
| cvssV3UserInteraction | String | The CVSS V3 user interaction (UI) of the vulnerability | Required |
| cvssV3Scope | String | The CVSS V3 scope (S) of the vulnerability | Unchanged |
| cvssV3Confidentiality | String | The CVSS V3 confidentiality (C) of the vulnerability | High |
| cvssV3Integrity | String | The CVSS V3 integrity (I) of the vulnerability | High |
| cvssV3Availability | String | The CVSS V3 availability (A) of the vulnerability | High |
| cvssV3Score | Double | The CVSS V3 score of the vulnerability | 8.8 |
| cvssV3Severity | String | The CVSS V3 severity of the vulnerability | High |
| cvssV3SeverityRank | Integer | The CVSS V3 severity rank of the vulnerability | 4 |
| skillLevel | String | The skill level of the vulnerability | unknown |
| skillLevelRank | Integer | The skill level rank of the vulnerability | 4 |
| severity | String | The severity of the vulnerability | Critical |
| severityRank | Integer | The severity rank of the vulnerability | 3 |
| severityScore | Integer | The severity score of the vulnerability | 9 |
| hasExploits | Boolean | Whether there are exploits associated with the vulnerability | false |
| threatFeedExists | Boolean | Whether a threat feed exists for the vulnerability | false |
| pciCompliant | Boolean | Whether the vulnerability is PCI compliant | false |
| pciSeverity | Integer | The PCI severity of the vulnerability | 5 |
| riskScore | Double | The real risk score of the vulnerability. Used only when riskScoreV2_0 is not present | 348.70102 |
| riskScoreV2_0 | Integer | The active risk score of the vulnerability | 589 |
| cves | List | An array of CVE IDs applicable to the vulnerability | [CVE-2023-29372] |
| dateAdded | Timestamp | Timestamp of when the vulnerability was added | 2023-06-13T00:00:00.000Z |
| dateModified | Timestamp | Timestamp of when the vulnerability was modified | 2023-06-15T00:00:00.000Z |
| datePublished | Timestamp | Timestamp of when the vulnerability was published | 2023-06-13T00:00:00.000Z |
| tags | List | The tags associated with the vulnerability | [Apple, Apple Mac OS X] |
| checkId | String | Unique identifier associated with the check | WINDOWS-HOTFIX-MS13-098-x64 |
| reintroducedTimestamp | Timestamp | Timestamp of when the vulnerability reappeared on the asset after previously being marked as remediated | 2023-06-15T00:00:00.000Z |
| epssscore | Double | Exploit Prediction Scoring System (EPSS) score is a data-driven probability score from 0 to 1 (or 0% to 100%) that a specific software vulnerability will be exploited in the wild within the next 30 days | 0.043800000 |
| epsspercentile | Double | The percentage of all CVEs that have an Exploit Prediction Scoring System (EPSS) score equal to or lower than this one. | 0.88546000 |
Vulnerability remediations export
vulnerability_remediation
| Field | Type | Definition | Example |
|---|---|---|---|
| orgId | String | Organization ID | a08de390-bb6a-4297-b1df-9ee58c7beb7a |
| assetId | String | Asset ID | 2b2b6a57-9136-4874-8af1-3f9f0a6de60a-default-asset-1 |
| cveId | String | CVE ID | CVE-YYYY-NNNNN |
| vulnId | String | Vulnerability ID | msft-cve-2023-29372 |
| proof | String | Proof describing how the asset is vulnerable to the vulnerability | <p><p>Vulnerable OS: Microsoft Windows Server 2012 R2 Standard Edition<p></p></p><p>Based on the following 3 results:<ol><li><p>Microsoft patch KB5027282 is not installed.</p></li><li><p>Microsoft patch KB5027271 is not installed.</p></li><li><p><ul><li>HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion<ul><li>UBR - contains unexpected value 20337</li></ul></li></ul></p></li></ol></p></p> |
| firstFoundTimestamp | Timestamp | Timestamp of when the vulnerability was found on the asset | 2023-11-06T11:40:37.000Z |
| reintroducedTimestamp | Timestamp | Timestamp of when the vulnerability was reintroduced on the asset | 2023-11-06T11:40:37.000Z |
| lastDetected | Timestamp | Timestamp of vulnerability remediation, determined by the last detection of the vulnerability on the asset. | 2023-11-06T11:40:37.000Z |
| lastRemoved | Timestamp | Timestamp of when the vulnerability was last removed from the asset | 2023-11-06T11:40:37.000Z |
| title | String | The title of the vulnerability | Microsoft CVE-2023-29372: Microsoft WDAC OLE DB provider for SQL Server Remote Code Execution Vulnerability |
| description | String | The description of the vulnerability, containing a mix of HTML/XML content | Microsoft CVE-2023-29372: Microsoft WDAC OLE DB provider for SQL Server Remote Code Execution Vulnerability |
| cvssV2Score | Double | The CVSS V2 score of the vulnerability | 8.8 |
| cvssV3Score | Double | The CVSS V3 score of the vulnerability | 8.8 |
| cvssV2Severity | String | The CVSS V2 severity of the vulnerability | High |
| cvssV3Severity | String | The CVSS V3 severity of the vulnerability | High |
| cvssV2AttackVector | String | The CVSS V2 attack vector (AV) of the vulnerability | Network |
| cvssV3AttackVector | String | The CVSS V3 attack vector (AV) of the vulnerability | Network |
| riskScoreV2_0 | Integer | The active risk score of the vulnerability | 589 |
| datePublished | Timestamp | Timestamp of when the vulnerability was published | 2023-06-13T00:00:00.000Z |
| dateAdded | Timestamp | Timestamp of when the vulnerability was added | 2023-06-13T00:00:00.000Z |
| dateModified | Timestamp | Timestamp of when the vulnerability was modified | 2023-06-15T00:00:00.000Z |
| epssscore | Double | Exploit Prediction Scoring System (EPSS) score is a data-driven probability score from 0 to 1 (or 0% to 100%) that a specific software vulnerability will be exploited in the wild within the next 30 days | 0.043800000 |
| epsspercentile | Double | The percentage of all CVEs that have an Exploit Prediction Scoring System (EPSS) score equal to or lower than this one. | 0.88546000 |