EDH - GCP Setup

The instructions below outline the steps required to complete the setup of GCP Event-driven Harvesting (EDH) in InsightCloudSec using the GCP Console and Cloud Shell.

Setup Overview

For InsightCloudSec to harvest data when triggered by events or changes in your GCP environment, you'll need to ensure that a service account is properly able to consume events, create a Pub/Sub topic, and create a Cloud Asset Inventory (CAI) feed. Review GCP's documentation for more information on these concepts. If setup properly, the following will occur:

1. GCP resource update events are captured by a CAI feed. Feeds support filtering per resource type.
2. The CAI feed routes events to a Pub/Sub topic and the topic ensures the events arrive in a paired Pub/Sub subscription.
3. InsightCloudSec monitors the subscription for new events and enqueues harvesters for the relevant resources.

Prerequisites

Before you configure anything in your GCP environment, you'll need the following:

• A functioning InsightCloudSec platform with the appropriate Admin (Org or Domain) permissions
• A GCP Organization added to InsightCloudSec (i.e., an existing GCP Service Account associated with InsightCloudSec)
  • Review Organizations (GCP) for more information
• A basic understanding of the relevant GCP services (Cloud Asset Inventory, Pub/Sub, Service Account)
• Appropriate GCP administrative permissions
  • The ability to create Pub/Sub topics within a project
  • The Cloud Asset Owner role has all the requisite Organization IAM permissions.
• At a minimum, the Cloud Asset Inventory API enabled within GCP
  • Some supported resources for EDH may require additional APIs; cross-reference the General GCP Recommended APIs with the list of Supported GCP EDH Resources/Asset Types

Step 1: Create a Pub/Sub Topic and Subscription

First, you'll have to create a matching Pub/Sub topic and subscription that will be used to publish and consume events. This topic and subscription pair needs to be created in the same project that contains the Service Account you created during the GCP setup.

1. Within the Google Cloud Console, open the project containing the service account that is currently integrated with InsightCloudSec.
   1. Navigate to Pub/Sub. If you've never used Pub/Sub for the project before, it may take a second to enable the API.
2. Click Create Topic.
   • For the Topic ID, type r7-gcp-edh.
   • Ensure Add a default subscription is selected. Once the topic is created, this option will ensure a matching subscription is also created with an ID of r7-gcp-edh-sub.
   • Click Create Topic.
3. After the topic is successfully created, copy the Subscription ID and keep it on hand for later.

Step 2: Update Service Account Permissions

Next, you must ensure the Service Account that will be used to consume events has the pubsub.subscriptions.consume permission to grab the events from the Pub/Sub topic. The easiest and best way to assure this is to assign the Service Account the Pub/Sub Subscriber role directly from the subscription itself. The same Service Account created during the GCP Organization integration process should be used for the GCP EDH setup.

1. Within the Google Cloud Console, from the Pub/Sub page, navigate to Subscriptions.
2. Select the subscription you just created in the previous section and expand the info panel (top right corner).
3. Click Add Principal.
4. On the Grant Access panel, update the service account with the proper permissions.
   1. Provide the service account name that's currently used by InsightCloudSec. It's easiest if you paste the full name into the field because GCP may not be able to auto-complete the value if you only type in a partial name.
   2. From the roles drop-down menu, filter for and select Pub/Sub Subscriber.
   3. Click Save.

Step 3: Create a CAI Feed

Before you can finish up the configuration within the InsightCloudSec UI, you'll need to create a CAI feed to send events to the topic. This section will assist in setting up a feed within the project that you created a topic/subscription in and is integrated with InsightCloudSec. Currently, feeds cannot be created via the GCP Console UI, so it will need to be created via the Cloud Shell, which is conveniently located in the GCP Console.

1. Within the Google Cloud Console, click the Activate Cloud Shell icon in the top right corner.

2. Copy the following code snippet and paste it into the Cloud Shell window, ensuring you replace:

   • <project_id> with the appropriate GCP Project ID
   • <topic_name> with the appropriate Pub/Sub topic name
   • <organization_id> with the appropriate GCP organization ID
   • The full list of asset types has been included in the example command below

   shell
   1
   gcloud asset feeds create r7-gcp-edh-resource-feed \
   2
   --pubsub-topic=projects/<project_id>/topics/<topic_name> \
   3
   --organization=<organization_id> \
   4
   --content-type=resource \
   5
   --asset-types=compute.googleapis.com/Autoscaler,bigtableadmin.googleapis.com/Instance,compute.googleapis.com/SslCertificate,file.googleapis.com/Instance,cloudfunctions.googleapis.com/CloudFunction,run.googleapis.com/Service,compute.googleapis.com/TargetVpnGateway,compute.googleapis.com/VpnTunnel,compute.googleapis.com/Image,container.googleapis.com/Cluster,compute.googleapis.com/Network,iam.googleapis.com/Role,secretmanager.googleapis.com/Secret,compute.googleapis.com/SecurityPolicy,iam.googleapis.com/ServiceAccountKey,compute.googleapis.com/Snapshot,compute.googleapis.com/Subnetwork,pubsub.googleapis.com/Subscription,pubsub.googleapis.com/Topic

3. Press Enter. If you have not used the CAI API before, you will be prompted to enable it and retry the command, then confirm this action.

4. Optionally, run the following command in the Cloud Shell window to verify the feed was created successfully, ensuring you replace <project_id> with the appropriate GCP Project ID:
   gcloud asset feeds list --project <project_id>

Step 4: Configure InsightCloudSec

Now that GCP has be configured for EDH, it's time to configure InsightCloudSec.

1. Login to your InsightCloudSec platform and click Cloud > Cloud Accounts in the left-hand navigation menu.
   1. Click EDH Consumers.
   2. Click Add EDH Configuration.
   3. From the drop-down menu, click GCP PubSub Consumer.
2. Update the configuration for the necessary information.
   1. Select the GCP Project (already onboarded in InsightCloudSec) that contains the service account & Pub/Sub topic and subscription.
   2. Provide the fully-qualified project subscription ID.
   3. Click Configure.

Post Setup Information

Congratulations on setting up Event Driven Harvesting for your GCP Organizations within InsightCloudSec. Below you'll find some important links about EDH in general.

• EDH Event Summaries and Reports
• Harvesting Failure Messages