GitLab DevOps Integration

The InsightCloudSec GitLab DevOps integration helps your security and development teams implement infrastructure-as-code (IaC) security and compliance scans (serviced using the CLI IaC Scanning Tool mimics) with GitLab.

Prerequisites

  • InsightCloudSec Domain Admin permissions
  • GitLab user with Maintainer permissions

Example repository structure

This page provides some example configuration based on the sample repository structure found in this section. There are two key directories in the sample repository:

  • iac_templates - Contains IaC templates for various providers and is sorted in to subdirectories accordingly.
  • pipeline_configurations- Stores the pipelines jobs according to their IaC driver. In this example, there are separate files based on whether or not the scan consumes an ICS IaC configuration or is configured with custom options (for example, a fixed list of desired insights). Explore the mimics documentation for configuration details.

You need to configure any example files found on this page to work with your unique environment, including where your IaC configurations reside, any defined environmental variables, whether or not Docker is used, and how the pipeline is triggered.

Example repository structure
bash
1
.
2
├── .gitlab-ci.yml
3
├── README.md
4
├── iac_templates
5
│   ├── .gitkeep
6
│   ├── cfn
7
│   │   ├── .gitkeep
8
│   │   └── my-cfn.yaml
9
│   └── terraform
10
│       ├── .gitkeep
11
│       └── my-tf.tf
12
└── pipeline_configurations
13
    ├── .gitkeep
14
    ├── cfn-remote-config-gitlab-ci.yaml
15
    ├── cfn-docker.yaml
16
    ├── terraform-docker.yaml
17
    └── terraform-remote-config-gitlab-ci.yaml

Example environment variables

In the example files on this page, you may find environment variables to keep sensitive data secure and to interact with InsightCloudSec:

  • BASE_URL - The web address of your InsightCloudSec environment, which can be found in System Settings.
  • ICS_API_KEY - API Key used to access the InsightCloudSec instance pointed to by BASE_URL. Used to retrieve an IaC configuration created in ICS.
  • ICS_CONFIG_NAME - Name of desired IaC configuration which defines the scanning behavior of mimics.
  • CFN_IAC_TEMPLATE_DIR_PATH - Path to directory containing Cloudformation (CFN) YAML templates.
  • TERRAFORM_IAC_TEMPLATE_DIR_PATH - Path to directory containing Terraform (TF) YAML templates.
  • TERRAFORM_VERSION - Version of terraform that should be downloaded and used to generate plans from any scanned Terraform files.

Integrate mimics into a GitLab pipeline

To integrate the InsightCloudSec (ICS) IaC scanning tool (mimics) with a GitLab CI/CD pipeline, you'll need to invoke mimics from a pipeline configuration YAML file. This file defines the stages and jobs that run against configured directories of IaC templates (such as Cloudformation or Terraform files). How mimics integrates with a pipeline depends on how you are currently using mimics. Currently, InsightCloudSec supports running mimics as an executable or using Docker. The only difference between the two approaches is the way in which the mimics scanning tool is obtained in the pipeline job.

GitLab Pipeline YAML structure and configuration

Each job triggers its corresponding pipeline file as indicated by the path in the include section of each pipeline configuration. In the example in this section, each of the jobs is triggered when there is a change to a configured directory defined in CFN_IAC_TEMPLATE_DIR_PATH. Any change to files in this directory (for example, edits or new files) will result in a being scan triggered and a pipeline job being created.

To configure a pipeline for a desired project, navigate to the GitLab Pipeline editor and follow the steps indicated.

Example pipeline configuration
yaml
1
stages:
2
  - triggers
3


4
trigger_cfn:
5
  stage: triggers
6
  trigger:
7
    include: pipeline_configurations/cfn-remote-config-gitlab-ci.yaml
8
  rules:
9
    changes:
10
      - $CFN_IAC_TEMPLATE_DIR_PATH/*
11


12
trigger_tf:
13
  stage: triggers
14
  trigger:
15
    include: pipeline_configurations/terraform-remote-config-gitlab-ci.yaml
16
  rules:
17
    changes:
18
      - $TERRAFORM_IAC_TEMPLATE_DIR_PATH/*
19

Visit the third-party vendor's documentation

For the most accurate information, we recommend that you visit the third-party vendor's product documentation:

Run mimics as an executable

The CLI IaC Scanning Tool can be run as an executable and downloaded directly into the GitLab pipeline to be run against the files to be scanned. The scanning tool's Insight configuration is pulled directly from your InsightCloudSec instance, so the following variables are required:

  • BASE_URL
  • ICS_API_KEY
  • ICS_CONFIG_NAME
Example CloudFormation configuration
yaml
1
stages:          
2
  - mimics_install
3
  - mimics_setup
4
  - mimics_scan
5


6
image: alpine
7


8
mimics-install-job:      
9
  stage: mimics_install
10
  script:
11
    - echo "Obtaining mimics binary..."
12
    - mkdir -p $CI_PROJECT_NAME/mimics-reports
13
    - wget https://artifacts.rapid7.com/cloudsec/mimics/latest/mimics_latest_linux_amd64 -O $CI_PROJECT_NAME/mimics
14
    - chmod +x $CI_PROJECT_NAME/mimics
15
  artifacts:
16
    paths:
17
      - $CI_PROJECT_NAME/mimics
18


19
mimics-setup-job:   
20
  stage: mimics_setup
21
  script:    
22
    - echo "Creating report directory..."
23
    - mkdir -p $CI_PROJECT_NAME/mimics-reports
24
  artifacts:
25
    paths:
26
      - $CI_PROJECT_NAME/mimics-reports
27


28
mimics-scan-job:
29
  stage: mimics_scan
30
  script:
31
    - echo "Initiating scan..."
32
    - $CI_PROJECT_NAME/mimics scan $CFN_IAC_TEMPLATE_DIR_PATH --api-key $ICS_API_KEY --base-url $ICS_BASE_URL --ics-config "$ICS_CONFIG_NAME" --report-path $CI_PROJECT_NAME/mimics-reports --report-formats all --report-name mimics_scan_ci_cd
33
  artifacts:
34
    when: always
35
    paths:
36
      - $CI_PROJECT_NAME/mimics-reports/mimics_scan_ci_cd*
Example Terraform configuration
yaml
1
stages:          
2
  - mimics_install
3
  - terraform_install
4
  - mimics_setup
5
  - terraform_setup
6
  - mimics_scan
7


8
image: alpine
9


10
variables:
11
    TERRAFORM_DOWNLOAD_URL: "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip"
12
    MIMICS_DOWNLOAD_URL: "https://artifacts.rapid7.com/cloudsec/mimics/latest/mimics_latest_linux_amd64"
13
    
14
mimics-install-job:      
15
  stage: mimics_install
16
  script:
17
    - echo "Obtaining mimics binary..."
18
    - mkdir -p $CI_PROJECT_NAME/mimics-reports
19
    - wget $MIMICS_DOWNLOAD_URL -O $CI_PROJECT_NAME/mimics
20
    - chmod +x $CI_PROJECT_NAME/mimics
21
  artifacts:
22
    paths:
23
      - $CI_PROJECT_NAME/mimics
24


25
terraform-install-job:
26
    stage: terraform_install
27
    script:
28
          - |
29
            echo "Downloading Terraform..."
30
            wget $TERRAFORM_DOWNLOAD_URL
31
            
32
            echo "Unzipping & placing Terraform in PATH..."
33
            unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip -d terraform_temp
34
            
35
            chmod +x terraform_temp/terraform
36
            mkdir -p $CI_PROJECT_NAME/tf-bin
37
            mv terraform_temp/terraform $CI_PROJECT_NAME/tf-bin/terraform
38
    artifacts:
39
      paths:
40
        - $CI_PROJECT_NAME/tf-bin
41


42
mimics-setup-job:   
43
  stage: mimics_setup
44
  script:    
45
    - echo "Creating report directory..."
46
    - mkdir -p $CI_PROJECT_NAME/mimics-reports
47
  artifacts:
48
    paths:
49
      - $CI_PROJECT_NAME/mimics-reports
50


51
terraform-setup-job:
52
    stage: terraform_setup
53
    before_script:
54
      - apk update
55
      - apk add jq
56
    script:
57
        - |
58
          echo "Copying binary to TF folder..."
59
          cp $CI_PROJECT_NAME/tf-bin/terraform $TERRAFORM_IAC_TEMPLATE_DIR_PATH/terraform
60
          
61
          echo "Moving to Terraform directory..."
62
          cd $TERRAFORM_IAC_TEMPLATE_DIR_PATH
63
          
64
          echo "Generating Terraform Plan..."
65
          ./terraform init
66
          ./terraform plan -out=tf.plan
67
          ./terraform show -json tf.plan > plan.json
68


69
          echo "Formatting Terraform Plan..."
70
          jq "." ./plan.json > formatted_plan.json
71
          mv formatted_plan.json plan.json
72
    artifacts:
73
      paths:
74
        - $TERRAFORM_IAC_TEMPLATE_DIR_PATH/plan.json
75


76
mimics-scan-job:
77
  stage: mimics_scan
78
  script:
79
    - echo "Initiating scan..."
80
    - $CI_PROJECT_NAME/mimics scan $TERRAFORM_IAC_TEMPLATE_DIR_PATH/plan.json --api-key $ICS_API_KEY --base-url $ICS_BASE_URL --ics-config "$ICS_CONFIG_NAME" --report-path $CI_PROJECT_NAME/mimics-reports --report-formats all --report-name mimics_scan_ci_cd
81
  artifacts:
82
    when: always
83
    paths:
84
      - $CI_PROJECT_NAME/mimics-reports/mimics_scan_ci_cd*

Run mimics using Docker

The CLI IaC Scanning Tool can be invoked with an InsightCloudSec Docker container accessed using Rapid7's public AWS Elastic Container Registry (ECR) repository. To do so, valid AWS credentials are required and must be exposed with environment variables for access in any jobs requiring the ECR repository:

  • AWS_ACCESS_KEY_ID - The ID of the Access Key associated with an IAM User to enable programmatic requests
  • AWS_SECRET_ACCESS_KEY - The secret value of the Access Key
Example CloudFormation Docker configuration
yaml
1
stages:          
2
  - docker_scan
3


4
image: docker:latest
5
services:
6
  - docker:dind
7


8
scan-job:      
9
  stage: docker_scan
10
  before_script:
11
    - apk update
12
    - apk add aws-cli
13


14
  script:
15
    - |
16
      aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID
17
      aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY
18
      
19
      mkdir -p ./$CI_PROJECT_NAME/mimics-reports
20
        
21
      docker run \
22
      -v ./$CFN_IAC_TEMPLATE_DIR_PATH:/data \
23
      -v ./$CI_PROJECT_NAME/mimics-reports:/mimics-reports \
24
      --name iac-mimics-container \
25
      public.ecr.aws/rapid7-insightcloudsec/ics/mimics:latest scan /data --api-key $ICS_API_KEY --base-url $ICS_BASE_URL --ics-config "$ICS_CONFIG_NAME" --report-path /mimics-reports --report-formats all --report-name mimics_scan_ci_cd
26
      
27
  artifacts:
28
    when: always
29
    paths:
30
      - ./$CI_PROJECT_NAME/mimics-reports/mimics_scan_ci_cd*
Example Terraform Docker configuration
yaml
1
stages:          
2
  - terraform_install
3
  - terraform_setup
4
  - docker_scan
5


6
image: docker:latest
7
services:
8
  - docker:dind
9


10
variables:
11
    TERRAFORM_DOWNLOAD_URL: "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip"
12


13
terraform-install-job:
14
    stage: terraform_install
15
    script:
16
          - |
17
            echo "Downloading Terraform..."
18
            wget $TERRAFORM_DOWNLOAD_URL
19
            
20
            echo "Unzipping & placing Terraform in PATH..."
21
            unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip -d terraform_temp
22
            
23
            chmod +x terraform_temp/terraform
24
            mkdir -p $CI_PROJECT_NAME/tf-bin
25
            mv terraform_temp/terraform $CI_PROJECT_NAME/tf-bin/terraform
26
    artifacts:
27
      paths:
28
        - $CI_PROJECT_NAME/tf-bin
29


30
terraform-setup-job:
31
    stage: terraform_setup
32
    before_script:
33
      - apk update
34
      - apk add jq
35
    script:
36
        - |
37
          echo "Copying binary to TF folder..."
38
          cp $CI_PROJECT_NAME/tf-bin/terraform $TERRAFORM_IAC_TEMPLATE_DIR_PATH/terraform
39
          
40
          echo "Moving to Terraform directory..."
41
          cd $TERRAFORM_IAC_TEMPLATE_DIR_PATH
42
          
43
          echo "Generating Terraform Plan..."
44
          ./terraform init
45
          ./terraform plan -out=tf.plan
46
          ./terraform show -json tf.plan > plan.json
47


48
          echo "Formatting Terraform Plan..."
49
          jq "." ./plan.json > formatted_plan.json
50
          mv formatted_plan.json plan.json
51
    artifacts:
52
      paths:
53
        - $TERRAFORM_IAC_TEMPLATE_DIR_PATH/plan.json
54


55


56
scan-job:      
57
  stage: docker_scan
58
  before_script:
59
    - apk update
60
    - apk add aws-cli
61


62
  script:
63
    - |
64
      aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID
65
      aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY
66
      
67
      mkdir -p ./$CI_PROJECT_NAME/mimics-reports
68
        
69
      docker run \
70
      -v ./$CFN_IAC_TEMPLATE_DIR_PATH:/data \
71
      -v ./$CI_PROJECT_NAME/mimics-reports:/mimics-reports \
72
      --name iac-mimics-container \
73
      public.ecr.aws/rapid7-insightcloudsec/ics/mimics:latest scan /data --api-key $ICS_API_KEY --base-url $ICS_BASE_URL --ics-config "$ICS_CONFIG_NAME" --report-path /mimics-reports --report-formats all --report-name mimics_scan_ci_cd
74
      
75
  artifacts:
76
    when: always
77
    paths:
78
      - ./$CI_PROJECT_NAME/mimics-reports/mimics_scan_ci_cd*

Analyzing scan results

Once a pipeline job has completed, its scan results are uploaded to GitLab as an artifact. Artifacts are uploaded at each stage of the pipeline and defined in each pipeline configuration using the artifacts variable.

To retrieve a pipeline artifact:

  1. Navigate to the desired GitLab Project.
  2. Under the Build menu, click Jobs.
  3. Select the job you wish to retrieve the artifacts for.
  4. Select Download in the Job Artifacts section of the job.