GitLab DevOps Integration

The InsightCloudSec GitLab DevOps integration helps your security and development teams implement infrastructure-as-code (IaC) security and compliance scans (serviced using the CLI IaC Scanning Tool mimics) with GitLab.

Prerequisites

InsightCloudSec Domain Admin permissions
GitLab user with Maintainer permissions

Example repository structure

This page provides some example configuration based on the sample repository structure found in this section. There are two key directories in the sample repository:

iac_templates - Contains IaC templates for various providers and is sorted in to subdirectories accordingly.
pipeline_configurations- Stores the pipelines jobs according to their IaC driver. In this example, there are separate files based on whether or not the scan consumes an ICS IaC configuration or is configured with custom options (for example, a fixed list of desired insights). Explore the mimics documentation for configuration details.

You need to configure any example files found on this page to work with your unique environment, including where your IaC configurations reside, any defined environmental variables, whether or not Docker is used, and how the pipeline is triggered.

Example repository structure


.
├── .gitlab-ci.yml
├── README.md
├── iac_templates
│   ├── .gitkeep
│   ├── cfn
│   │   ├── .gitkeep
│   │   └── my-cfn.yaml
│   └── terraform
│       ├── .gitkeep
│       └── my-tf.tf
└── pipeline_configurations
    ├── .gitkeep
    ├── cfn-remote-config-gitlab-ci.yaml
    ├── cfn-docker.yaml
    ├── terraform-docker.yaml
    └── terraform-remote-config-gitlab-ci.yaml

Example environment variables

In the example files on this page, you may find environment variables to keep sensitive data secure and to interact with InsightCloudSec:

BASE_URL - The web address of your InsightCloudSec environment, which can be found in System Settings.
ICS_API_KEY - API Key used to access the InsightCloudSec instance pointed to by BASE_URL. Used to retrieve an IaC configuration created in ICS.
ICS_CONFIG_NAME - Name of desired IaC configuration which defines the scanning behavior of mimics.
CFN_IAC_TEMPLATE_DIR_PATH - Path to directory containing Cloudformation (CFN) YAML templates.
TERRAFORM_IAC_TEMPLATE_DIR_PATH - Path to directory containing Terraform (TF) YAML templates.
TERRAFORM_VERSION - Version of terraform that should be downloaded and used to generate plans from any scanned Terraform files.

Integrate mimics into a GitLab pipeline

To integrate the InsightCloudSec (ICS) IaC scanning tool (mimics) with a GitLab CI/CD pipeline, you’ll need to invoke mimics from a pipeline configuration YAML file. This file defines the stages and jobs that run against configured directories of IaC templates (such as Cloudformation or Terraform files). How mimics integrates with a pipeline depends on how you are currently using mimics. Currently, InsightCloudSec supports running mimics as an executable or using Docker. The only difference between the two approaches is the way in which the mimics scanning tool is obtained in the pipeline job.

GitLab Pipeline YAML structure and configuration

Each job triggers its corresponding pipeline file as indicated by the path in the include section of each pipeline configuration. In the example in this section, each of the jobs is triggered when there is a change to a configured directory defined in CFN_IAC_TEMPLATE_DIR_PATH. Any change to files in this directory (for example, edits or new files) will result in a being scan triggered and a pipeline job being created.

To configure a pipeline for a desired project, navigate to the GitLab Pipeline editor and follow the steps indicated.

Example pipeline configuration


stages:
  - triggers
 
trigger_cfn:
  stage: triggers
  trigger:
    include: pipeline_configurations/cfn-remote-config-gitlab-ci.yaml
  rules:
    changes:
      - $CFN_IAC_TEMPLATE_DIR_PATH/*
 
trigger_tf:
  stage: triggers
  trigger:
    include: pipeline_configurations/terraform-remote-config-gitlab-ci.yaml
  rules:
    changes:
      - $TERRAFORM_IAC_TEMPLATE_DIR_PATH/*

Visit the third-party vendor’s documentation

For the most accurate information, we recommend that you visit the third-party vendor’s product documentation:

GitLab CI/CD Pipelines: https://docs.gitlab.com/ee/ci/pipelines/
GitLab CI/CD YAML syntax reference: https://docs.gitlab.com/ee/ci/yaml/

Run mimics as an executable

The CLI IaC Scanning Tool can be run as an executable and downloaded directly into the GitLab pipeline to be run against the files to be scanned. The scanning tool’s Insight configuration is pulled directly from your InsightCloudSec instance, so the following variables are required:

BASE_URL
ICS_API_KEY
ICS_CONFIG_NAME

Example CloudFormation configuration


stages:          
  - mimics_install
  - mimics_setup
  - mimics_scan
 
image: alpine
 
mimics-install-job:      
  stage: mimics_install
  script:
    - echo "Obtaining mimics binary..."
    - mkdir -p $CI_PROJECT_NAME/mimics-reports
    - wget https://artifacts.rapid7.com/cloudsec/mimics/latest/mimics_latest_linux_amd64 -O $CI_PROJECT_NAME/mimics
    - chmod +x $CI_PROJECT_NAME/mimics
  artifacts:
    paths:
      - $CI_PROJECT_NAME/mimics
 
mimics-setup-job:   
  stage: mimics_setup
  script:    
    - echo "Creating report directory..."
    - mkdir -p $CI_PROJECT_NAME/mimics-reports
  artifacts:
    paths:
      - $CI_PROJECT_NAME/mimics-reports
 
mimics-scan-job:
  stage: mimics_scan
  script:
    - echo "Initiating scan..."
    - $CI_PROJECT_NAME/mimics scan $CFN_IAC_TEMPLATE_DIR_PATH --api-key $ICS_API_KEY --base-url $ICS_BASE_URL --ics-config "$ICS_CONFIG_NAME" --report-path $CI_PROJECT_NAME/mimics-reports --report-formats all --report-name mimics_scan_ci_cd
  artifacts:
    when: always
    paths:
      - $CI_PROJECT_NAME/mimics-reports/mimics_scan_ci_cd*

Example Terraform configuration


stages:          
  - mimics_install
  - terraform_install
  - mimics_setup
  - terraform_setup
  - mimics_scan
 
image: alpine
 
variables:
    TERRAFORM_DOWNLOAD_URL: "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip"
    MIMICS_DOWNLOAD_URL: "https://artifacts.rapid7.com/cloudsec/mimics/latest/mimics_latest_linux_amd64"
    
mimics-install-job:      
  stage: mimics_install
  script:
    - echo "Obtaining mimics binary..."
    - mkdir -p $CI_PROJECT_NAME/mimics-reports
    - wget $MIMICS_DOWNLOAD_URL -O $CI_PROJECT_NAME/mimics
    - chmod +x $CI_PROJECT_NAME/mimics
  artifacts:
    paths:
      - $CI_PROJECT_NAME/mimics
 
terraform-install-job:
    stage: terraform_install
    script:
          - |
            echo "Downloading Terraform..."
            wget $TERRAFORM_DOWNLOAD_URL
            
            echo "Unzipping & placing Terraform in PATH..."
            unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip -d terraform_temp
            
            chmod +x terraform_temp/terraform
            mkdir -p $CI_PROJECT_NAME/tf-bin
            mv terraform_temp/terraform $CI_PROJECT_NAME/tf-bin/terraform
    artifacts:
      paths:
        - $CI_PROJECT_NAME/tf-bin
 
mimics-setup-job:   
  stage: mimics_setup
  script:    
    - echo "Creating report directory..."
    - mkdir -p $CI_PROJECT_NAME/mimics-reports
  artifacts:
    paths:
      - $CI_PROJECT_NAME/mimics-reports
 
terraform-setup-job:
    stage: terraform_setup
    before_script:
      - apk update
      - apk add jq
    script:
        - |
          echo "Copying binary to TF folder..."
          cp $CI_PROJECT_NAME/tf-bin/terraform $TERRAFORM_IAC_TEMPLATE_DIR_PATH/terraform
          
          echo "Moving to Terraform directory..."
          cd $TERRAFORM_IAC_TEMPLATE_DIR_PATH
          
          echo "Generating Terraform Plan..."
          ./terraform init
          ./terraform plan -out=tf.plan
          ./terraform show -json tf.plan > plan.json
 
          echo "Formatting Terraform Plan..."
          jq "." ./plan.json > formatted_plan.json
          mv formatted_plan.json plan.json
    artifacts:
      paths:
        - $TERRAFORM_IAC_TEMPLATE_DIR_PATH/plan.json
 
mimics-scan-job:
  stage: mimics_scan
  script:
    - echo "Initiating scan..."
    - $CI_PROJECT_NAME/mimics scan $TERRAFORM_IAC_TEMPLATE_DIR_PATH/plan.json --api-key $ICS_API_KEY --base-url $ICS_BASE_URL --ics-config "$ICS_CONFIG_NAME" --report-path $CI_PROJECT_NAME/mimics-reports --report-formats all --report-name mimics_scan_ci_cd
  artifacts:
    when: always
    paths:
      - $CI_PROJECT_NAME/mimics-reports/mimics_scan_ci_cd*

Run mimics using Docker

The CLI IaC Scanning Tool can be invoked with an InsightCloudSec Docker container accessed using Rapid7’s public AWS Elastic Container Registry (ECR) repository. To do so, valid AWS credentials are required and must be exposed with environment variables for access in any jobs requiring the ECR repository:

AWS_ACCESS_KEY_ID - The ID of the Access Key associated with an IAM User to enable programmatic requests
AWS_SECRET_ACCESS_KEY - The secret value of the Access Key

Example CloudFormation Docker configuration


stages:          
  - docker_scan
 
image: docker:latest
services:
  - docker:dind
 
scan-job:      
  stage: docker_scan
  before_script:
    - apk update
    - apk add aws-cli
 
  script:
    - |
      aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID
      aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY
      
      mkdir -p ./$CI_PROJECT_NAME/mimics-reports
        
      docker run \
      -v ./$CFN_IAC_TEMPLATE_DIR_PATH:/data \
      -v ./$CI_PROJECT_NAME/mimics-reports:/mimics-reports \
      --name iac-mimics-container \
      public.ecr.aws/rapid7-insightcloudsec/ics/mimics:latest scan /data --api-key $ICS_API_KEY --base-url $ICS_BASE_URL --ics-config "$ICS_CONFIG_NAME" --report-path /mimics-reports --report-formats all --report-name mimics_scan_ci_cd
      
  artifacts:
    when: always
    paths:
      - ./$CI_PROJECT_NAME/mimics-reports/mimics_scan_ci_cd*

Example Terraform Docker configuration


stages:          
  - terraform_install
  - terraform_setup
  - docker_scan
 
image: docker:latest
services:
  - docker:dind
 
variables:
    TERRAFORM_DOWNLOAD_URL: "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip"
 
terraform-install-job:
    stage: terraform_install
    script:
          - |
            echo "Downloading Terraform..."
            wget $TERRAFORM_DOWNLOAD_URL
            
            echo "Unzipping & placing Terraform in PATH..."
            unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip -d terraform_temp
            
            chmod +x terraform_temp/terraform
            mkdir -p $CI_PROJECT_NAME/tf-bin
            mv terraform_temp/terraform $CI_PROJECT_NAME/tf-bin/terraform
    artifacts:
      paths:
        - $CI_PROJECT_NAME/tf-bin
 
terraform-setup-job:
    stage: terraform_setup
    before_script:
      - apk update
      - apk add jq
    script:
        - |
          echo "Copying binary to TF folder..."
          cp $CI_PROJECT_NAME/tf-bin/terraform $TERRAFORM_IAC_TEMPLATE_DIR_PATH/terraform
          
          echo "Moving to Terraform directory..."
          cd $TERRAFORM_IAC_TEMPLATE_DIR_PATH
          
          echo "Generating Terraform Plan..."
          ./terraform init
          ./terraform plan -out=tf.plan
          ./terraform show -json tf.plan > plan.json
 
          echo "Formatting Terraform Plan..."
          jq "." ./plan.json > formatted_plan.json
          mv formatted_plan.json plan.json
    artifacts:
      paths:
        - $TERRAFORM_IAC_TEMPLATE_DIR_PATH/plan.json
 
 
scan-job:      
  stage: docker_scan
  before_script:
    - apk update
    - apk add aws-cli
 
  script:
    - |
      aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID
      aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY
      
      mkdir -p ./$CI_PROJECT_NAME/mimics-reports
        
      docker run \
      -v ./$CFN_IAC_TEMPLATE_DIR_PATH:/data \
      -v ./$CI_PROJECT_NAME/mimics-reports:/mimics-reports \
      --name iac-mimics-container \
      public.ecr.aws/rapid7-insightcloudsec/ics/mimics:latest scan /data --api-key $ICS_API_KEY --base-url $ICS_BASE_URL --ics-config "$ICS_CONFIG_NAME" --report-path /mimics-reports --report-formats all --report-name mimics_scan_ci_cd
      
  artifacts:
    when: always
    paths:
      - ./$CI_PROJECT_NAME/mimics-reports/mimics_scan_ci_cd*

Analyzing scan results

Once a pipeline job has completed, its scan results are uploaded to GitLab as an artifact. Artifacts are uploaded at each stage of the pipeline and defined in each pipeline configuration using the artifacts variable.

To retrieve a pipeline artifact:

Navigate to the desired GitLab Project.
Under the Build menu, click Jobs.
Select the job you wish to retrieve the artifacts for.
Select Download in the Job Artifacts section of the job.