Job Backlog Export

In the System Administration section of InsightCloudSec, customers can view important System Health Information like the Current Job Backlog. The Current Job Backlog is a great way to understand the current load on your environment. Exporting these Job Backlog metrics can be used to drive improvements from a scaling perspective and provide historical data for up to 15 months. InsightCloudSec provides support for export of Job Backlog metrics to AWS through CloudWatch and Google Cloud Platform through Stackdriver.

Product name to be replaced

You may observe that some components, screen captures, or examples use our former product name, DivvyCloud. This doesn't affect the configuration or the product's functionality, and we will notify you as we replace these component names.

AWS CloudWatch Job Backlog Export

The following instructions are for setting up export for AWS and CloudWatch.

Required Permissions

Update the IAM policy for your account to allow for "cloudwatch:PutMetricData".

Setting Up Export

  1. From the Administration Section of the left-side panel navigation, select System Administration > System and scroll to the Job Backlog Settings.
  2. Under Target Account, Admins can select the AWS Account that will receive exported job backlog information.
    • You will need the name of the target account and target region for export.
    • You also have the option to use instance authentication if you're using Instance Assume Role. For most cases, you will not need to use instance authentication.
    • To include a custom name, fill in the Target Namespace (AWS Only) field, otherwise this will display as InsightCloudSec in CloudWatch.

Viewing Details in the AWS Console

In the AWS console, this information will show up in your Custom Namespaces under Metrics. You will have many options for how this data will be displayed.

  • You have the choice of viewing time series data from 1 hour up to 15 months.
  • In the example below, entries are being pushed every 5 minutes but that can be modified.

Cloudwatch Metrics

GCP Stackdriver Job Backlog Export

The following instructions are for setting up export for GCP and Stackdriver.

Prerequisites (GCP)

In order for InsightCloudSec to provide the appropriate details, you will need:

  • A connected GCP cloud
  • A GCP role with appropriate permissions

To enable the Stackdriver Monitoring API, complete the following steps to enable or confirm.

  1. In the GCP Console, access the main left-panel navigation menu. Under API & Services select Library.
  2. Search for Stackdriver and choose the option for Stackdriver API.
  3. From the Stackdriver API summary page, select Enable.
    • If these settings are already enabled, you will see a green tick and a Manage option and no further action is required.

Setting Up Export (in InsightCloudSec)

  1. From the Administration Section of the left-side panel navigation, select System Administration > System and scroll to the Job Backlog Settings.
  2. Under Target Account, select the GCP Account from the drop-down that will receive exported job backlog information; save your selection.

Viewing Details in the GCP Console

  1. From the main GCP console, search for the Stackdriver service, select Stackdriver from the dropdown result, and click to Go to Stackdriver.
  2. From the left panel, click Dashboards, click Create Dashboard and select Add Chart.
  3. Give your chart a useful title. Then in the Find resource type and metric field, search for divvy to find these three metrics:
    • job_backlog_standard
    • job_backlog_high_priority
    • job_backlog_immediate
  4. Select your desired metric, click Add Metric. Repeat this for each one and then select Save.
  5. Verify that your data displays appropriately. This can take up to a minute or more.