Azure LPA Setup
The Azure LPA feature can be used to harvest and send detailed data about Azure users and roles to your InsightCloudSec instance. If you have any issues or questions with this setup, reach out to the support team through the Customer Support Portal.
The Azure LPA feature relies on infrastructure deployed in the user’s account to collect action data and aggregate action statistics. This configuration is performed through the Azure LPA configuration library. For an overview of this infrastructure, review Azure LPA design. You can deploy the necessary infrastructure automatically using the Azure LPA configuration library contained with the InsightCloudSec Azure LPA repository.
To setup Azure LPA, you'll need to complete the following steps:
- Step 1: Download and Setup the Repository -- Download and setup the repository then deploy the Azure LPA infrastructure.
- Step 2: InsightCloudSec Configuration -- Setup Azure LPA data collection within InsightCloudSec and begin receiving data.
Before configuring Azure LPA, you'll need the following:
- MacOS or Linux local environment
- Admin access to the Azure subscription that contains the LPA data
- The Azure subscription to associate with LPA is connected to InsightCloudSec
- You are logged in to the associated Azure subscription via the Azure CLI with appropriate Admin permissions
Azure CloudShell is not supported!
The instructions below are currently only supported in local MacOS or Linux environment or in a virtual machine.
Step 1: Download and Setup the Repository
The repository is publicly available in an S3 bucket.
Value Names (DivvyCloud vs. InsightCloudSec)
Some components use our former product name (DivvyCloud vs. InsightCloudSec). Updates to the naming of these components will be communicated when changes are made, but note that the name difference does not affect setup or functionality within the product.
- Download the repository using the following link: https://s3.amazonaws.com/get.divvycloud.com/prodserv/ics-azure-lpa/ics-azure-lpa-latest.zip
- Extract the repository's contents to the desired location.
Upon extracting the repository, you should examine its contents for basic understanding. Below is an outline of the sections of this repository:
README.md-- contains extensive documentation for the Azure LPA Python Library
roles-- Azure role definitions for the roles to be deployed to your subscription
src/ics_azure_lpa/automation-- Azure deployment automation framework
- Automation entails the creation of the Azure LPA infrastructure, addition/removal of subscriptions, and destruction of the Azure LPA infrastructure
src/ics_azure_lpa/client-- Azure client interaction
- Client interaction relates to either direct interactions with the Kusto/Azure Data Explorer (ADX) cluster or Azure Storage (to receive the final data products)
arm_templates-- Azure ARM templates
bicep_templates-- Azure Bicep templates
tests/integration-- Integration tests (long-running tests that manipulate actual resources).
tests/regression-- Regression tests (short-running tests that ensure code consistency).
Azure LPA deployment is best done via the Python deployment script that uses Bicep, the Azure CLI, and the Kusto client. Setting up a Python virtual environment specifically (e.g.,
pyenv virtualenv ...) for the Azure LPA deployment is recommended.
This library was developed and tested using Python 3.8.5, but newer versions may work as well.
- From a terminal, navigate to the repository.
- Before continuing, we highly recommend you review the
- Before continuing, we highly recommend you review the
- Install dependencies:
make deps-py-dev(development dependencies) or
make deps-py(build dependencies)
- Create or refresh the Azure ARM templates:
- Deploy the infrastructure:
Additional information about this command can be viewed by running the following:
- Answer the interactive prompts to ensure proper deployment location and options. An example run of the deployment script is included below:
Deployment of the entire Azure LPA infrastructure can take anywhere from 40 to 60 minutes.
1(divvycloud-azure-lpa) admin@my-macbook divvycloud-azure-lpa % deployLpa2Deployment location (Central US, East US, East US 2, US Gov Iowa, US Gov Virginia, North Central US, South Central US, West US, North Europe, West Europe, East Asia, Southeast Asia, Japan East, Japan West, Brazil South, Australia East, Australia Southeast, Central India, South India, West India) [East US]:3With virtual network [y/N]: y45Starting LPA deployment for Resource Group: "my-resource-group"!672022-05-31 13:35:11,052 - divvy_azure_lpa.automation.accounts - INFO - Subscription: "Azure Alpha" [23456a7b-234b-2345-a23b-a23456b7c89d] Tenant [12345a6b-123b-1234-a12b-a12345b6c78d] (accounts.py:48)82022-05-31 13:35:11,055 - divvy_azure_lpa.automation.accounts - INFO - Subscription: "qa-testbed" [34567a8b-3451-3456-a34b-a34567b8c90d] Tenant [12345a6b-123b-1234-a12b-a12345b6c78d] (accounts.py:48)92022-05-31 13:35:11,055 - divvy_azure_lpa.automation.accounts - INFO - Default subscription: "qa-testbed" [34567a8b-3451-3456-a34b-a34567b8c90d] Tenant [12345a6b-123b-1234-a12b-a12345b6c78d] (accounts.py:61)102022-05-31 13:35:11,914 - divvy_azure_lpa.automation.lpa - INFO - Looking for Azure Event Grid System Topic: "Microsoft.Resources.Subscriptions" within subscription [34567a8b-3451-3456-a34b-a34567b8c90d]... (lpa.py:1012)112022-05-31 13:35:13,212 - divvy_azure_lpa.automation.lpa - INFO - Utilizing Azure Event Grid System Topic: "Microsoft.Resources.Subscriptions" (Name: "subscription-resource-events" RG: "r7-ics-lpa" Sub: [34567a8b-3451-3456-a34b-a34567b8c90d]) (lpa.py:396)122022-05-31 13:35:13,213 - divvy_azure_lpa.automation.lpa - INFO - Target LPA RG: "my-resource-group" Sub: [34567a8b-3451-3456-a34b-a34567b8c90d] Tenant: [12345a6b-123b-1234-a12b-a12345b6c78d] (lpa.py:404)132022-05-31 13:35:14,262 - divvy_azure_lpa.automation.lpa - INFO - Deployment configuration chosen: Virtual Network (lpa.py:219)142022-05-31 13:35:14,262 - divvy_azure_lpa.automation.lpa - INFO - Attempting to deploy/update deployment associated with: "templates/vn/overallLPA.bicep"... (lpa.py:230)152022-05-31 13:35:14,263 - divvy_azure_lpa.automation.lpa - WARNING - Deployment can take around 60 minutes. (lpa.py:234)162022-05-31 14:03:55,929 - divvy_azure_lpa.automation.bicep - INFO - Provisioning state: Succeeded (bicep.py:90)172022-05-31 14:03:55,929 - divvy_azure_lpa.automation.network_security_group - INFO - Attempting to deploy/update deployment associated with: "templates/vn/SecurityGroupAllowlist.bicep"... (network_security_group.py:151)182022-05-31 14:04:32,305 - divvy_azure_lpa.automation.bicep - INFO - Provisioning state: Succeeded (bicep.py:90)192022-05-31 14:04:32,466 - divvy_azure_lpa.automation.lpa - INFO - Opening Virtual Network up to local access (temporarily...) (lpa.py:579)202022-05-31 14:04:44,393 - divvy_azure_lpa.automation.lpa - INFO - Updating Kusto cluster to allow client access for principal "bc9bf403-9b66-4be5-8a4d-c6ff399440f5"... (lpa.py:591)212022-05-31 14:04:46,067 - divvy_azure_lpa.automation.lpa - INFO - Kusto cluster access updated. (lpa.py:602)222022-05-31 14:04:46,067 - divvy_azure_lpa.automation.lpa - INFO - Closing Virtual Network to local access. (lpa.py:608)232022-05-31 14:04:57,976 - divvy_azure_lpa.automation.lpa - INFO - Enabling workflow: "workflow"... (lpa.py:617)242022-05-31 14:04:59,156 - divvy_azure_lpa.automation.lpa - INFO - Workflow enabled. (lpa.py:624)252022-05-31 14:04:59,156 - divvy_azure_lpa.automation.lpa - INFO - Deployment complete for Resource Group: "my-resource-group"! (lpa.py:626)2627Deployment of resource group "my-resource-group" complete!2829To access data via CLI, set the environment variables:30export AZURE_STORAGE_CONNECTION_STRING="REDACTED"31export AZURE_STORAGE_CONTAINER_NAME="events"32Invoke: downloadLpaData
A successful deployment yields three crucial values for the next section:
- Resource group name
- Azure Storage connection string
- Azure Storage container name
At this point, the Azure LPA infrastructure is deployed and the data collection is beginning. Data is collected continually, but aggregation of the LPA action statistics happens at midnight.
Step 2: InsightCloudSec Configuration
Now that the Azure LPA infrastructure has been deployed, it's time to enable data collection within InsightCloudSec.
Login to your InsightCloudSec platform and click the settings button (cog icon) in the top-right corner.
Click IAM Settings>Azure LPA Settings. The LPA Settings page will appear with a list of Azure Subscriptions that have been added to InsightCloudSec.
Next to the proper subscription, click the vertical three dots to open the menu.
Click Enable LPA.
Provide the values for harvesting the LPA data.
- Provide the resource group name where the Azure LPA infrastructure is deployed (created in step 1)
- Provide the connection string for the Azure Storage resource that hosts the Azure LPA data (created in step 1)
- Provide the container name inside the Azure Storage resource that contains the Azure LPA data (created in step 1)
Click OK. You'll be returned to the LPA Settings page with the Status column updated to reflect LPA's current state. The status will typically be Enabled, but relatively recent deployments may reflect that the data is still awaiting aggregation. Review LPA States for more information.
Once LPA is successfully enabled and data has been collected, review Azure LPA Usage for information on viewing your data.
It can take up to 24 hours for data to be harvested and appear in InsightCloudSec.
During deployment, the current Azure command line interface (CLI) user may not have the necessary permissions to create the Azure LPA infrastructure. Alternatively, you may want to create a specialized deployment user that is relatively de-privileged to execute the deployment commands. The Azure LPA infrastructure library provides the command
createLpaDeploymentRole that allows for the creation of a custom role that can be used to perform the Azure LPA infrastructure deployment.
User Loses Connection Information
While the user should retain the key three values (resource group name, connection string, container name) listed in the terminal post-deployment, it is possible they may lose that data. They would need to resurrect that data via the Azure Console:
- To find the resource group name, you'll need to search through your resource groups with the Feature tag set to LPA.
- To find the Azure Storage account name, examine the LPA-related resource group (step 1).
- The connection string and container name can be reached from navigating to the Azure Storage account (step 2) via Azure Storage Explorer.
On the LPA Settings page, there are statuses associated with each Azure subscription added to InsightCloudSec and its LPA state:
|Enabled||LPA is enabled and configured properly & data is available|
|Not Enabled||LPA has not been enabled for the subscription yet|
|Error||Invalid connection information|
|Collecting Data||Waiting on data|
The Collecting Data state indicates that the initial collection (scheduled daily) has not yet occurred.
Additional Deployment Script Options
After downloading the repository and installing up any dependencies, the Azure LPA configuration library is available for use. While deploying the LPA infastructure is its main function, there are some other important functions available.
After you have successfully deployed the Azure LPA infrastructure, you may want to manually download the data being collected.
Before you can manually download Azure LPA data, you will need the following on hand:
- Open a terminal locally and set the following environment variables (or pass them in via the command line):
- Run the following command and look in the set download path for the data:
You have the option of using the same Azure LPA deployment infrastructure against multiple subscriptions within the same Azure tenant. This will save you money, as the expensive pieces will be re-used instead of duplicated.
The use of the
subscribeLpa command will allow the linkage of other subscriptions to existing Azure LPA infrastructure. Invoking the command will result in an interactive dialog that allows registering another subscription.
Upon completion of this subscription addition process within the Azure LPA library, you can reuse the same resource group name, connection string, and container name with the ICS configuration process for each of these existing, linked accounts.
If you'd like to remove a subscription from Azure LPA harvesting, you can use the
If you'd like to remove the infrastructure associated with the Azure LPA feature from your Azure subscription, you can use the