Specification File

So you want to build a plugin? You’ve come to the right place!

The following will demonstrate how to correctly build a plugin using the Python SDK.

Plugins support any number of triggers or actions. Plugins are defined using a plugin.spec.yaml, and a plugin shell can be codegen’d using

insight-plugin create <path/to/plugin.spec.yaml>

When writing plugins, work directly out of the plugins repository. Locate your plugins.spec.yaml to plugins/myplugin.yaml and generate the skeleton from there, so it fits nicely with the other plugins.

Plugin Specification

Plugins are code generated from a plugin spec file.

The Plugin Specification

First, create a file called .yaml and populate it with some data:

yaml
1
plugin_spec_version: v2
2
extension: plugin
3
products: [insightconnect]
4
name: example
5
title: Example Plugin
6
description: Example plugin for testing
7
version: 1.0.0
8
supported_versions: ["1.0.0", "1.0.1"]
9
vendor: rapid7
10
status: []
11
tags: [malware, hash]
12
hub_tags:
13
  use_cases: [data_enrichment,threat_detection_and_response]
14
  keywords: [malware, hash]
15
  features: []
16
resources:
17
  source_url: https://github.com/rapid7/insightconnect-plugins/tree/master/example
18
  license_url: https://github.com/rapid7/insightconnect-plugins/blob/master/LICENSE
19
  vendor_url: https://www.rapid7.com
20
enable_cache: true
21

22
types:
23
  person:
24
    first_name:
25
      title: First Name
26
      type: string
27
    last_name:
28
      title: Last Name
29
      type: string
30

31
connection:
32
  hostname:
33
    title: Host
34
    description: Enter the hostname
35
    type: string
36
    required: true
37
    example: https://www.google.com/
38
  port:
39
    title: Port
40
    description: Enter the port
41
    type: integer
42
    default: 80
43
    required: true
44
    example: 8080
45
  username:
46
    title: Username
47
    description: Enter the username
48
    type: string
49
    required: true
50
    example: UserName
51

52
triggers:
53
  emit_greeting:
54
    title: Trigger a New Greeting
55
    description: Triggers a greeting every interval
56
    input:
57
      interval:
58
        title: Interval
59
        description: How frequently (in seconds) to trigger a greeting
60
        type: integer
61
        default: 15
62
        required: true
63
        example: 20
64
    output:
65
      greeting:
66
        title: Greeting
67
        description: The user generated greeting message
68
        type: person
69
        required: true
70
        example: Hello World!
71

72
actions:
73
  say_goodbye:
74
    title: Say Goodbye
75
    description: Emit say goodbye message
76
    input:
77
      name:
78
        title: Name
79
        description: Name to say goodbye to
80
        type: string
81
        required: true
82
        example: Rapid7Name
83
    output:
84
      message:
85
        title: Message
86
        description: User generated message
87
        type: string
88
        required: true
89
        example: Hello World!

Let's review this file in detail.

Metadata Section

At the top of the spec, you define metadata about your plugin such as the name, version, and tags. The metadata section ends with two newline characters.

The table below lists all the metadata properties.

Below, you can find more details about each property.

plug_spec_version

This defines the plugin specification version, which changes how plugins operate.

• v2 - is a standalone plugin as an HTTP REST server that is invoked once. This is currently the only supported plugin_spec_version in InsightConnect.
• v1 - is a standalone plugin that exits after each invocation (legacy, and unsupported in insightConnect).

extension

This defines the type of extension. For plugins, it will be:

extension: plugin

More extension types may be available in the future.

products

This field defines compatible Insight products for categorization in The Extension Library.

For plugins, it will be:

products: [insightconnect]

name

The name of the plugin for development purposes. This value determines the directory and file system naming of the plugin.

E.g. name: example_plugin

See the style guide on how to properly set the name.

title

The title of the plugin is displayed in the InsightConnect UI. Users of the product see this value. The title should conform to proper noun rules.

E.g. title: Example Plugin

description

The InsightConnect UI displays the plugin's description. Users of the product see this value. The description should conform to sentence-like structure.

E.g. description: Example plugin for testing

version

The plugin's version follows the semver format.

• Plugins in development should begin with 0.1.0
• Plugins in production should begin with 1.0.0

supported_versions

This lists versions of the vendor software the plugin maintainers support, and against which the plugin has been tested. The values should be in ascending order.

• Supported versions are strings that identify specific releases of the vendor product software. The format of the version numbers should match the format used by the product vendor.
• For SaaS software without specific versions, supported versions should be a running list of timestamps that this plugin was tested against the vendor API in the format yyyy-MM-dd.

E.g. ['1.0.0', '1.0.1', 1.2.3']

E.g. ['2021-01-31]

vendor

The vendor who developed the plugin, used for organization, searching, and display on The Extension Library

tags

Optional tags that describe your plugin's functionality. The InsightConnect UI and The Extension Library uses these tags to find your plugin.

Tags are created as an array of strings.

E.g. tags: [ malware, hash ]

hub_tags

Tags that describe your plugin's use cases, features, and searchable keywords for The Extension Library.

Tags are created as an array of strings.

yaml
1
hub_tags:
2
  use_cases: [data_enrichment,threat_detection_and_response]
3
  keywords: [malware, hash]
4
  features: []

The following table lists valid use_cases values:

resources

Resources provides a place to link official URLs in The Extension Library.

An example for an open-source plugin on Github:

yaml
1
resources:
2
  source_url: https://github.com/rapid7/insightconnect-plugins/tree/master/example
3
  license_url: https://github.com/rapid7/insightconnect-plugins/blob/master/LICENSE
4
  vendor_url: http://www.rapid7.com

If a key doesn't apply, then the key should be omitted in the spec.

support

Field to determine the ownership of the plugin for maintenance as well as to handle user requests.

Values can be: rapid7, partner, community.

E.g. support: community

status

Status describes the state of the integration as an array of strings. Therefore multiple statuses can be applied at the same time.

E.g. status: [deprecated]

Values are deprecated, insecure, and obsolete.

If none of these apply, leave the array empty:

E.g. status: []

enable_cache

Enables persistent storage on the host for a given plugin.

E.g. enable_cache: true

Base Types

The following types are available in the types, connection, trigger input/output, and action input/output sections.

The base type must be selected from one of the valid data types below:

You can also specify a collection (array) type by doing []<base type> wrapped in strings, e.g. type: "[]string". You must wrap it in double-quotes.

enum is an optional modifier that can be used to limit and declare the possible options for an input field. For example:

yaml
1
actions:
2
  forward:
3
    title: Forward Lookup
4
    description: Forward DNS query
5
    input:
6
      domain:
7
        type: string
8
        description: Domain name to resolve
9
        required: true
10
      query:
11
        type: string
12
        description: Optional query type e.g. ANY, A, MX, NS, etc.
13
        enum:
14
        - A
15
        - AAA
16
        - ANY
17
        - CNAME
18
        - MX
19
        - NS
20
        - PTR
21
        - SOA

The enumerated DNS query types are selectable from a drop-down box in the InsightConnect Workflow UI.

Custom Types Section

The types section defines any custom complex type objects you may need. It is in the format of a map which has the <type name> identifier as the key, and then an object definition that describes the property for the object. You can use these types later in your plugin specification as types for your input/output parameters, or within your connection definition.

The table below lists all of the types properties.

yaml
1
types:
2
  <type name>:
3
    <parameter no. 1 identifier>:
4
      type: <choose from a valid type>
5
      title: <optional, descriptive name for the UI>
6
      description: <optional, string description>
7
      default: <optional,  default>
8
      required: <optional, true or false, default is false. Set to true if required>
9
   <parameter no. 2 identifier>:
10
      type: <choose from a valid type>
11
      title: <optional, descriptive name for the UI>
12
      description: <optional, string description>
13
      default: <optional,  default>
14
      required: <optional, true or false, default is false. Set to true if required>

In the example at the beginning of this page, we defined a custom type called person that has two properties: first_name and last_name.

These types will now be available in the connection sections, trigger input/output, and action input/output sections to use in addition to the base types.

yaml
1
...
2
   output:
3
     greeting:
4
       title: Greeting
5
       description: Greeting for person
6
       required: true
7
       type: person

Connection Section

The connection section defines the configuration variables you need for a connection. If your plugin does not require a connection, you can remove this section.

If a connection does exist, the UI will render a Configure Connection page for the values in the Workflow Builder.

In the connection section, each variable is defined in a map with its identifier. The identifier must be unique, including lowercase letters, numbers, or underscores only.

The table below lists all of the connection properties.

yaml
1
connection:
2
  <parameter no. 1 identifier>:
3
    title: <descriptive name for parameter>
4
    description: <tooltip description string>
5
    type: <choose from a valid type>
6
    required: <optional, true or false, default is false. Set to true if required>
7
    default: <optional, default>
8
    example: <optional, sample value that would be valid>
9
  <parameter no. 2 identifier>:
10
    title: <descriptive name for parameter>
11
    description: <tooltip description string>
12
    type: <choose from a valid type>
13
    required: <optional, true or false, default is false. Set to true if required>
14
    default: <optional, default>
15
    example: <optional, sample value that would be valid>

Input and output sections should not contain name identifiers.

Triggers Section

Triggers are long-running processes that kick off workflows. All workflows must start with a trigger. This is in contrast to actions, which must occur after a trigger in a workflow. These actions are short-lived processes that run, do something, and then stop.

The code for triggers run in the body of a continuous loop, where the purpose is to poll for new events. When this event is found, the plugin triggers a workflow with the found event as JSON passes it to the next plugin in the workflow.

Example:

1. Poll an RSS feed for new content
2. When new content is available, pass the content as JSON to the engine
3. Sleep for x seconds
4. Repeat

A plugin can define 0 or more triggers. Simply create a section triggers and add a map (defined by a unique key) for each of the triggers you support.

The table below is a list of all the trigger properties.

yaml
1
triggers:
2
  <unique trigger identifier>:
3
    title: <optional, descriptive name for the UI>
4
    description: <optional, string description>
5
    input: <optional, map of identifier -> variable inputs>
6
    output: <map of identifier -> variable outputs>

Input and output sections should not contain name identifiers. For usability, all triggers should support an input called frequency, which is passed to a sleep mechanism in the SDK.

yaml
1
frequency:
2
  type: integer
3
  description: Poll frequency in seconds
4
  default: 300
5
  required: false

Actions Section

Each plugin may contain zero or more actions, which are short-running processes that execute, return results, and then exit. Workflows typically contain many actions chained together. Plugin code is executed in a Docker container. As the action initializes, the connection object is created and then the action code runs.

The table below lists all of the actions properties.

yaml
1
actions:
2
  <unique action identifier>:
3
    title: <optional, descriptive name for the UI>
4
    description: <optional, string description>
5
    input: <optional, map of identifier -> variable inputs>
6
    output: <map of identifier -> variable outputs>

UI Rendering

The UI renders the name, title, description, etc. properties from the spec file in the following ways.

Action & Trigger Selection

Action & Trigger Configuration

Creating a Plugin

Now that you have a valid plugin spec file, you can codegenerate a plugin shell. You need to have the insight-plugin tool. The latest tooling is available here.

Run it as follows:

insight-plugin create plugin.spec.yaml

You may change plugin.spec.yaml to the path of your plugin spec. By default, it will generate the plugin folder with the value of name defined in your plugin spec yaml. You can also use the -d or --target-dir flag to specify a custom path to generate your plugin e.g. --target-dir /tmp/example.

Our plugin above was generated in /tmp/example. You should be able to build it now by going to the folder and typing make.

 
1
$ insight-plugin create plugin.spec.yaml -d /tmp/example
2
INFO[0000] Writing to file: /tmp/example/komand_example/__init__.py
3
INFO[0000] Writing to file: /tmp/example/komand_example/util/__init__.py
4
INFO[0000] Writing to file: /tmp/example/Makefile
5
INFO[0000] Writing to file: /tmp/example/.gitignore
6
INFO[0000] Writing to file: /tmp/example/setup.py
7
INFO[0000] Writing to file: /tmp/example/Dockerfile
8
INFO[0000] Writing to file: /tmp/example/requirements.txt
9
INFO[0000] Writing to file: /tmp/example/bin/komand_example
10
INFO[0000] Writing to file: /tmp/example/komand_example/connection/connection.py
11
INFO[0000] Writing to file: /tmp/example/komand_example/connection/__init__.py
12
INFO[0000] Writing to file: /tmp/example/komand_example/connection/schema.py
13
INFO[0000] Writing to file: /tmp/example/komand_example/actions/say_goodbye/action.py
14
INFO[0000] Writing to file: /tmp/example/komand_example/actions/say_goodbye/schema.py
15
INFO[0000] Writing to file: /tmp/example/komand_example/actions/say_goodbye/__init__.py
16
INFO[0000] Writing to file: /tmp/example/komand_example/actions/__init__.py
17
INFO[0000] Writing to file: /tmp/example/komand_example/triggers/emit_greeting/trigger.py
18
INFO[0000] Writing to file: /tmp/example/komand_example/triggers/emit_greeting/schema.py
19
INFO[0000] Writing to file: /tmp/example/komand_example/triggers/emit_greeting/__init__.py
20
INFO[0000] Writing to file: /tmp/example/komand_example/triggers/__init__.py
21
INFO[0000] Code generation completed! Your plugin is located at: /tmp/example

The make task does two things: Creates a <vendor>/<name>:<version> container with the plugin code, and also a <vendor>-<name>-<version>.tar.gz package that can be uploaded via the plugins UI within the InsightConnect interface.

All the plugins are built as Docker containers for easy packaging and running.

You can always type make during the process of writing your plugin to build the latest version of it. Any code changes require packaging for use, so get in the habit of running make often.

You now have a plugin shell you can interact with. To continue, see the next page on Running Plugins.