Skip to Content
Insightcloudsec- Azure DevOps Integration

Azure DevOps Integration

The InsightCloudSec Azure DevOps integration helps your security and development teams integrate infrastructure-as-code (IaC) security and compliance scans (serviced using the CLI IaC Scanning Tool mimics) with Azure Pipelines and Boards. Setting up this integration is as simple as copying a YAML file to your code repository!

Prerequisites

  • InsightCloudSec Domain Admin permissions

HTML viewer extension recommended

Though it’s not required, InsightCloudSec highly recommends including a third-party HTML viewer extension  as part of your Azure DevOps organization as it can improve the IaC output and pipeline artifacts viewing experience.

Integrate with Azure Pipelines

To integrate the InsightCloudSec CLI IaC Scanning Tool (mimics) with Azure Pipelines, you’ll need to incorporate one of the example YAML files included in the following sections into your code repository. The file you use depends on how you are currently using mimics. Currently, InsightCloudSec supports running mimics as an executable or using Docker. In each example YAML file, we have included the following Azure Pipeline environment variables to keep sensitive data secure and interact with InsightCloudSec and AWS Elastic Container Registry (ECR) if applicable:

  • BASE_URL - The web address of your InsightCloudSec environment, which can be found in System Settings.
  • ICS_API_KEY - An API Key used to access InsightCloudSec data without traditional username-password authentication
    • We recommend this value remains a secret within Azure DevOps
  • IAC_TEMPLATE_DIR_PATH - The file path to the directory that contains your IaC templates (relative to the project root). For example, /path/to/cloudformation.
  • IAC_CONFIG_NAME - The name of the IaC Configuration  to run the scan against
    • If your configuration name contains spaces, you will need to wrap the environment variable value in quotation marks. For example, "Test Configuration".

For more information on Azure Pipelines variables, see Define Variables .

⚠️

Planning on storing sensitive information within Azure DevOps environment variables?

Azure DevOps environment variables can be stored as secrets to not expose any sensitive information, such as API or access keys. When configuring environment variables for your Azure Pipeline, ensure that sensitive data is not exposed by selecting the Keep this a secret checkbox when inputting variables.

Run as an executable

The CLI IaC Scanning Tool can be run as an executable using Terraform or Amazon Web Services’ (AWS) CloudFormation.

Using Terraform

trigger: 
  branches:
    include: 
    - releases/*
 
pool:
  vmImage: "ubuntu-latest" # Define the virtual machine image to use for the pipeline
 
variables:
  terraformVersion: "1.9.2" # Define the Terraform version to ensure consistency across runs
  terraformDownloadUrl: "https://releases.hashicorp.com/terraform/$(terraformVersion)/terraform_$(terraformVersion)_linux_amd64.zip" # URL for Terraform download
  mimicsDownloadUrl: "https://artifacts.rapid7.com/cloudsec/mimics/latest/mimics_latest_linux_amd64" # URL for Mimics download
 
jobs:
  - job: Rapid7_IaC_Terraform_Scanner
    displayName: "Rapid7 IaC scanner"
 
    steps:
      # Check if Terraform is already installed, and download it if necessary
      - script: |
          if [ ! -f /usr/local/bin/terraform ]; then
              wget $(terraformDownloadUrl)
          else
              mv /usr/local/bin/terraform /usr/local/bin/terraform.old
              wget $(terraformDownloadUrl)
          fi
 
          # Unzip and install Terraform
          unzip "terraform_$(terraformVersion)_linux_amd64.zip" -d terraform_temp
          chmod +x terraform_temp/terraform
          mv terraform_temp/terraform /usr/local/bin/
        displayName: "Install Terraform"
 
      # Initialize Terraform, create a plan, and convert it to JSON
      - script: |
          terraform init
          terraform plan -out=tf.plan
          terraform show -json tf.plan > plan.json
        displayName: "Initialize Terraform & Generate Terraform plan"
        workingDirectory: $(System.DefaultWorkingDirectory)$(IAC_TEMPLATE_DIR_PATH)
 
      # Format plan.json to provide position output of violating resources
      - script: |
          jq "." ./plan.json > formatted_plan.json
          mv formatted_plan.json plan.json
        displayName: "Format Terraform Plan to provide accurate output"
        workingDirectory: $(System.DefaultWorkingDirectory)$(IAC_TEMPLATE_DIR_PATH)
 
       # Download the IaC Scanning Tool Mimics executable
      - script: |
          wget $(mimicsDownloadUrl) -O $(Agent.ToolsDirectory)/mimics
          chmod +x $(Agent.ToolsDirectory)/mimics
        displayName: "Install Mimics-binary" # Downloads and installs the Mimics tool (binary executable) for infrastructure as code (IaC) scanning
 
      # Use the Mimics executable to scan the plan.json
      - script: |
          mkdir -p $(System.DefaultWorkingDirectory)/mimics-reports
          $(Agent.ToolsDirectory)/mimics scan \
            $(System.DefaultWorkingDirectory)$(IAC_TEMPLATE_DIR_PATH)/plan.json \
            --api-key $(ICS_API_KEY) \
            --base-url $(BASE_URL) \
            --no-verify \
            --ics-config $(IAC_CONFIG_NAME) \
            --log-format json \
            --report-formats all \
            --report-name results-rapid7_iac \
            --report-path "$(System.DefaultWorkingDirectory)/mimics-reports" \
            --save-report \
            --no-fail \
            --verbose
        displayName: "Scan IaC files with Mimics"
 
      # Publish additional files (results-rapid7_iac artifacts)
      - task: PublishBuildArtifacts@1
        displayName: "Publish Scan Artifacts"
        inputs:
          pathtoPublish: "$(System.DefaultWorkingDirectory)/mimics-reports"
          artifactName: "results-rapid7_iac" # Publishes the scan reports as build artifacts for later access
 
      # Publish the HTML report using the PublishHtmlReport task
      - task: PublishHtmlReport@1
        condition: succeededOrFailed()
        inputs:
          reportDir: $(System.DefaultWorkingDirectory)/mimics-reports/results-rapid7_iac.html
          tabName: "R7 IaC Scan Results" # Publishes the HTML scan report to the Azure DevOps build summary for easy access

Using CloudFormation

trigger: 
  branches:
    include: 
    - releases/*
 
pool:
  vmImage: "ubuntu-latest" # Define the virtual machine image to use for the pipeline
 
variables:
  mimicsDownloadUrl: "https://artifacts.rapid7.com/cloudsec/mimics/latest/mimics_latest_linux_amd64" # URL for Mimics download
 
jobs:
  - job: Rapid7_IaC_CloudFormation_Scanner
    displayName: "Rapid7 IaC scanner"
 
    steps:
       # Download the IaC Scanning Tool Mimics executable
      - script: |
          wget $(mimicsDownloadUrl) -O $(Agent.ToolsDirectory)/mimics
          chmod +x $(Agent.ToolsDirectory)/mimics
        displayName: "Install Mimics-binary" # Downloads and installs the Mimics tool (binary executable) for infrastructure as code (IaC) scanning
 
      # Use the Mimics executable to scan the plan.json
      - script: |
          mkdir -p $(System.DefaultWorkingDirectory)/mimics-reports
          $(Agent.ToolsDirectory)/mimics scan \
            $(System.DefaultWorkingDirectory)$(IAC_TEMPLATE_DIR_PATH) \
            --api-key $(ICS_API_KEY) \
            --base-url $(BASE_URL) \
            --no-verify \
            --ics-config $(IAC_CONFIG_NAME) \
            --log-format json \
            --report-formats all \
            --report-name results-rapid7_iac \
            --report-path "$(System.DefaultWorkingDirectory)/mimics-reports" \
            --save-report \
            --no-fail \
            --verbose
        displayName: "Scan IaC files with Mimics"
 
      # Publish additional files (results-rapid7_iac artifacts)
      - task: PublishBuildArtifacts@1
        displayName: "Publish Scan Artifacts"
        inputs:
          pathtoPublish: "$(System.DefaultWorkingDirectory)/mimics-reports"
          artifactName: "results-rapid7_iac" # Publishes the scan reports as build artifacts for later access
 
      # Publish the HTML report using the PublishHtmlReport task
      - task: PublishHtmlReport@1
        condition: succeededOrFailed()
        inputs:
          reportDir: $(System.DefaultWorkingDirectory)/mimics-reports/results-rapid7_iac.html
          tabName: "R7 IaC Scan Results" # Publishes the HTML scan report to the Azure DevOps build summary for easy access

Run using Docker

The CLI IaC Scanning Tool can also be run as a Docker container using Terraform or AWS CloudFormation, but requires additional configuration. To run mimics in a Docker container, you’ll need to provide the following variables to authenticate for the AWS CLI so InsightCloudSec can retrieve the mimics container image from AWS Elastic Container Registry (ECR):

  • AWS_ACCESS_KEY_ID - The ID of the Access Key associated with an IAM User to enable programmatic requests
  • AWS_SECRET_ACCESS_KEY - The secret value of the Access Key

To view all publicly-available mimics container images, visit the Rapid7 ECR Gallery . For more information on AWS Access Keys and long-term credentials, see Managing access keys for IAM users .

⚠️

AWS authentication should be stored as a secret

Both variables should be stored as secrets within your Azure DevOps Pipeline.

Using Terraform

trigger: 
  branches:
    include: 
    - releases/*
 
pool:
  vmImage: "ubuntu-latest" # Define the virtual machine image to use for the pipeline
 
 
variables:
  terraformVersion: "1.9.2" # Define the Terraform version to ensure consistency across runs
  terraformDownloadUrl: "https://releases.hashicorp.com/terraform/$(terraformVersion)/terraform_$(terraformVersion)_linux_amd64.zip" # URL for Terraform download
 
jobs:
- job: Rapid7_Docker_IaC_CloudFormation_Scanner
  displayName: "Rapid7 Docker IaC scanner"
  steps:
 
  # Configure AWS CLI with access key, secret key, and region
  - script: |
      aws configure set aws_access_key_id $(AWS_ACCESS_KEY_ID)
      aws configure set aws_secret_access_key $(AWS_SECRET_ACCESS_KEY)
    displayName: "Configuring AWS CLI Credentials"
    
  # Check if Terraform is already installed, and download it if necessary
  - script: |      
      if [ ! -f /usr/local/bin/terraform ]; then
          wget $(terraformDownloadUrl)
      else
          mv /usr/local/bin/terraform /usr/local/bin/terraform.old
          wget $(terraformDownloadUrl)
      fi
      
      # Unzip and install Terraform
      unzip terraform_$(terraformVersion)_linux_amd64.zip -d terraform_temp
      chmod +x terraform_temp/terraform
      mv terraform_temp/terraform /usr/local/bin/
    displayName: "Install Terraform"
 
  # Initialize Terraform, create a plan, and convert it to JSON
  - script: |
      terraform init
      terraform plan -out tf.plan
      terraform show -json tf.plan > plan.json
    displayName: "Initialize Terraform & Generate Terraform plan"
    workingDirectory: $(System.DefaultWorkingDirectory)$(IAC_TEMPLATE_DIR_PATH)
 
  # Format plan.json to provide position output of violating resources
  - script: |
      jq '.' ./plan.json > formatted_plan.json
      mv formatted_plan.json plan.json
    displayName: "Format Terraform Plan to provide accurate output"
    workingDirectory: $(System.DefaultWorkingDirectory)$(IAC_TEMPLATE_DIR_PATH)
 
  # Scan the provided IaC Template with the IaC mimics image
  - script: |
      pull(){
        mkdir -p $(System.DefaultWorkingDirectory)/data/mimics-reports
        docker run \
        -v $(System.DefaultWorkingDirectory)$(IAC_TEMPLATE_DIR_PATH):/data \
        -e MIMICS_BASE_URL=$(BASE_URL) \
        -e MIMICS_API_KEY=$(ICS_API_KEY) \
        --name iac-mimics-container \
        public.ecr.aws/rapid7-insightcloudsec/ics/mimics:latest scan /data/plan.json --no-verify --ics-config $(IAC_CONFIG_NAME) --log-format json --report-formats all --report-name results-rapid7_iac --report-path "/data/mimics-reports" --save-report --no-fail --verbose
      }
 
      echo '1 2 3 5 8 13' | tr ' ' '\n' | while read t; do
        pull && break
        echo >&2 "docker pull failed; sleeping $t seconds"
        sleep $t
      done  
    displayName: "Scan IaC files with Mimics"
 
  # Copy mimics reports from docker container to host
  - script: |
      docker cp iac-mimics-container:data/mimics-reports $(System.DefaultWorkingDirectory)/data
    displayName: Copy Mimics Reports from Docker Container to Host
 
  # Publish additional files (results-rapid7_iac artifacts)
  - task: PublishBuildArtifacts@1
    displayName: "Publish Scan Artifacts"
    inputs:
      pathtoPublish: "$(System.DefaultWorkingDirectory)/data/mimics-reports"
      artifactName: "results-rapid7_iac"
 
  # Publish the HTML report using the PublishHtmlReport task
  - task: PublishHtmlReport@1
    displayName: "Publish HTML Report"
    condition: succeededOrFailed()
    inputs:
      reportDir: $(System.DefaultWorkingDirectory)/data/mimics-reports/results-rapid7_iac.html
      tabName: "R7 IaC Scan Results"

Using CloudFormation

trigger: 
  branches:
    include: 
    - releases/*
 
pool:
  vmImage: "ubuntu-latest" # Define the virtual machine image to use for the pipeline
 
jobs:
- job: Rapid7_Docker_IaC_Terraform_Scanner
  displayName: "Rapid7 Docker IaC scanner"
  steps:
 
  # Configure AWS CLI with access key, secret key, and region
  - script: |
      aws configure set aws_access_key_id $(AWS_ACCESS_KEY_ID)
      aws configure set aws_secret_access_key $(AWS_SECRET_ACCESS_KEY)
    displayName: "Configuring AWS CLI Credentials"
 
  # Scan the provided IaC Template with the IaC mimics image
  - script: |
      pull(){
        mkdir -p $(System.DefaultWorkingDirectory)/data/mimics-reports
        # Use Docker to run the Mimics tool for scanning
        docker run \
        -v $(System.DefaultWorkingDirectory)$(IAC_TEMPLATE_DIR_PATH):$(IAC_TEMPLATE_DIR_PATH) \
        -e MIMICS_BASE_URL=$(BASE_URL) \
        -e MIMICS_API_KEY=$(ICS_API_KEY) \
        --name iac-mimics-container \
        public.ecr.aws/rapid7-insightcloudsec/ics/mimics:latest scan $(IAC_TEMPLATE_DIR_PATH) --no-verify --ics-config $(IAC_CONFIG_NAME) --log-format json --report-formats all --report-name results-rapid7_iac --report-path "/data/mimics-reports" --save-report --no-fail --verbose
      }
 
      echo '1 2 3 5 8 13' | tr ' ' '\n' | while read t; do
        pull && break
        echo >&2 "docker pull failed; sleeping $t seconds"
        sleep $t
      done  
    displayName: "Scan IaC files with Mimics"
 
  # Copy mimics reports from docker container to host
  - script: |
      docker cp iac-mimics-container:data/mimics-reports $(System.DefaultWorkingDirectory)/data
    displayName: Copy Mimics Reports from Docker Container to Host
 
  # Publish additional files (results-rapid7_iac artifacts)
  - task: PublishBuildArtifacts@1
    displayName: "Publish Scan Artifacts"
    inputs:
      pathtoPublish: "$(System.DefaultWorkingDirectory)/data/mimics-reports"
      artifactName: "results-rapid7_iac"
 
  # Publish the HTML report using the PublishHtmlReport task
  - task: PublishHtmlReport@1
    displayName: "Publish HTML Report"
    condition: succeededOrFailed()
    inputs:
      reportDir: $(System.DefaultWorkingDirectory)/data/mimics-reports/results-rapid7_iac.html
      tabName: "R7 IaC Scan Results"

Integrate with Azure Boards (optional)

To integrate the InsightCloudSec CLI IaC Scanning Tool (mimics) with Azure Boards, you’ll need to follow the instructions for integrating Azure Pipelines first and then incorporate the example in this section with the appropriate YAML file for your setup. In the following example YAML section, we have included the following environment variables to assist your new InsightCloudSec Azure Pipelines integration in interacting with Azure Boards:

  • PAT - A Personal Access Token (PAT) for your Azure DevOps user, which helps pipelines create work items for each finding produced by the IaC analysis scans.
    • PATs should be stored as a secret within your Azure DevOps environment variables. For more information on PATs, visit Use Personal Access Tokens .
  • AZURE_BOARDS_ORGANIZATION - The Azure DevOps Organization where the Azure Board work items should be placed.
  • AZURE_BOARDS_PROJECT - The Azure DevOps Project where Azure Board work items should be placed.

Example Azure Board script for Azure Pipeline integration

The following example section needs to be placed in the appropriate section of the example Azure Pipelines YAML file. The location is determined by how you are running mimics:

  • If you are running mimics using an executable, copy and paste it after the Scan IaC files with Mimics step
  • If you are running mimics using a Docker container, copy and paste it after the Copy Mimics Reports from Docker Container to Host step
# Create Azure Devops work items for each finding produced by the IaC Mimics scan
- script: |
    # Construct the URL for API requests, choosing the correct work item type and API version.
    url="https://dev.azure.com/$(AZURE_BOARDS_ORGANIZATION)/$(AZURE_BOARDS_PROJECT)/_apis/wit/workitems/\$Issue?api-version=6.0"
 
    # Specify the path to the SARIF file generated by the previous steps
    sarifFile="$(System.DefaultWorkingDirectory)/mimics-reports/results-rapid7_iac.sarif"
 
    # Check for the existence of the SARIF file and process it
    if [ -f "$sarifFile" ]; then
      echo "Found SARIF file: $sarifFile"
 
      # Iterate through each result in the SARIF file, extracting relevant details
      jq -c '.runs[].results[]' "$sarifFile" | while read -r result; do
        ruleId=$(jq -r ".ruleId" <<< $result)
        message=$(jq -r ".message.text" <<< $result)
        locations=$(jq -r '.locations[]' <<< $result)
        title=$(jq -c --arg ruleId "$ruleId" '.runs[].tool.driver.rules[] | select(.id == $ruleId).name' "$sarifFile")
 
        # Prepare a JSON payload for creating a new work item via the Azure DevOps REST API
        json=$(jq --arg ruleId "$ruleId" --arg message "$message" --arg title "$title" --arg project "$(AZURE_BOARDS_PROJECT)" '
          [
            {"op": "add", "path": "/fields/System.Title", "value": $title},
            {"op": "add", "path": "/fields/System.Description", "value": "<div>\($message)</div><pre style=\"white-space: pre-line\">\(.properties.changes)</pre>"},
            {"op": "add", "path": "/fields/System.State", "value": "To Do"},
            {"op": "add", "path": "/fields/System.AreaPath", "value": $project}
          ]
        ' <<< $locations)
 
        # Make an API call to create a new work item with the prepared payload
        curl -X POST -H "Content-Type: application/json-patch+json" -H "Authorization: Basic $(echo -n ":$PAT" | base64)" -d "$json" "$url"
      done
    else
      echo "SARIF file not found: $sarifFile"
    fi
  displayName: "Process SARIF and Create Azure Board Work Items"
  env:
    PAT: $(PAT) # Pass the Personal Access Token (PAT) securely as an environment variable

Create a Azure Pipeline

After you have selected the appropriate example YAML file for your project and copied it to the code repository, you are ready to set up the integration in Azure Pipelines.

Something not quite right?

For the most up-to-date information, refer to Microsoft Azure’s documentation on creating an Azure Pipeline. 

  1. Log in to the Azure Console.
  2. Navigate to Azure Pipelines.
  3. Click New Pipeline.
  4. In the Connect section, follow the steps to connect and select the host of your code repository.
  5. In the Configure section, click Starter Pipeline.
  6. Copy and paste the example YAML file you selected in the Integrate with Azure Pipelines over the existing YAML on this page.
  7. Click Variables and provide the applicable environment variables.
  8. Validate the YAML and update as necessary. For example, setting pipeline triggers  and a valid Terraform version.
  9. Click Run > Save.