Download Log

The “GET Download Logs” request method allows you to download and stream log events to your machine for the given log IDs and query parameters over HTTPS.

This API call returns results in Response Codes and Response Headers.

Authentication

Before you start, make sure that you have an API key with Read/Write privileges.

URL

Use the following URL format for your API call: https://REGION.rest.logs.insight.rapid7.com/download/logs/:log_id(s)

Make sure to replace REGION with your location:

  • US
  • EU
  • CA
  • AU
  • AP

URL Path Parameters

To call the logs you want to download, you need to list the individual log IDs separated by a colon. For example: aee00b66-a543-43dc-b093-53963c2e8f41:d8eacea6-3dbd-4163-8fc2-3ef5067bd7c9:1c1f2885-ed93-4650-9e14-d46af4bf0886

A functional URL with log IDs might look like this:

1
https://REGION.rest.logs.insight.rapid7.com/download/logs/aee00b66-a543-43dc-b093-53963c2e8f41:d8eacea6-3dbd-4163-8fc2-3ef5067bd7c9:1c1f2885-ed93-4650-9e14-d46af4bf0886
2

Query Parameters

Use the following parameters to build an API GET query:

Parameter

Type

Required

Default

Description

from

Long

optional (if time_range is supplied)

N/A

Start timestamp in milliseconds.

to

Long

optional

Current system time

End timestamp in milliseconds.

time_range

String

optional (if from is supplied)

N/A

The relative time range in a readable format.

See Relative Time Support for more information.

query

String

optional

where(error)

The LEQL query to match desired log events. Do not use a calculation

limit

Long

optional

20 million

Max number of log events to download; cannot exceed 20 million.

Log download limit

You can download a maximum of 10 logs, or 20 million logs events, as indicated query parameter.

Response

After you submit the query, the API Response is a stream of log events with response codes and response headers following the HTTP chunked transfer encoding format.

Response Codes

A “HTTP 200” response means the request is valid.

A “HTTP 400” indicates the request is not valid due to an error in one of the following:

  • invalid logs
  • invalid time range query parameters
  • invalid LEQL query
  • invalid limit (0 is the only invalid limit)

A “HTTP 404” response indicates the authentication headers have the wrong API key.

Response Headers

The API response will also include response headers such as the following:

  • Transfer-Encoding → chunked
  • Content-Type → text/plain; charset=UTF-8
  • Content-Disposition → attachment; filename=.log
  • R7-RateLimit-LimitBytes → max downloadable bytes limit for a rate limit period
  • R7-RateLimit-RemainingBytes → remaining downloadable bytes limit for the rate limit period

Rate Limiting

Requests are subject to rate limits within a 15 minute period, per API key on an API endpoint.

We can just say here that rate limit is:

Time window 15 mins Max of Download requests 150 And maximum size of download (GB) 75 GB

Downloaded bytes rate limit in a rate limiting period cannot exceed the value in R7-RateLimit-LimitBytes header.

The bytes for the rate limit period is indicated by R7-RateLimit-RemainingBytes header.

Sample Call

A functional API call to download logs might look like the following:

json
1
import requests
2
import json
3
import time
4
5
API_KEY = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
6
LOG_ID = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
7
8
def handle_response(resp):
9
response = resp
10
time.sleep(1)
11
if response.status_code == 200:
12
print response.content
13
else:
14
print response.status_code
15
16
def make_request(provided_url=None):
17
headers = {'x-api-key': API_KEY}
18
19
url = "https://rest.logentries.com/download/logs/%s" % LOG_ID
20
params = {"time_range": "last 20 mins"}
21
req = requests.get(url, headers=headers, params=params)
22
return req
23
24
def get_log():
25
req = make_request()
26
handle_response(req)
27
28
def start():
29
get_log()
30
31
if __name__ == '__main__':
32
start()