Plugin Spec

So you want to build a plugin? You’ve come to the right place!

The following will demonstrate how to correctly build a plugin using the Python SDK.

Plugins support any number of triggers or actions. Plugins are defined using a plugin.spec.yaml, and a plugin shell can be codegen’d using

icon-plugin generate python <path/to/plugin.spec.yaml>

When writing plugins, work directly out of the plugins repository. Locate your plugins.spec.yaml to plugins/myplugin.yaml and generate the skeleton from there so it fits nicely with the other plugins.

Plugin Specification

Plugins are code generated from a plugin spec file.

The Plugin Specification

First, create a file called example.yaml and populate it with some data:

yaml
1
plugin_spec_version: v2
2
extension: plugin
3
products: [insightconnect]
4
name: example
5
title: Example Plugin
6
description: Example plugin for testing
7
version: 1.0.0
8
vendor: rapid7
9
status: []
10
tags: [malware, hash]
11
hub_tags:
12
use_cases: [data_enrichment,threat_detection_and_response]
13
keywords: [malware, hash]
14
features: []
15
resources:
16
source_url: https://github.com/rapid7/insightconnect-plugins/tree/master/example
17
license_url: https://github.com/rapid7/insightconnect-plugins/blob/master/LICENSE
18
vendor_url: http://www.rapid7.com
19
enable_cache: true
20
21
types:
22
person:
23
first_name:
24
title: First Name
25
type: string
26
last_name:
27
title: Last Name
28
type: string
29
30
connection:
31
hostname:
32
title: Host
33
description: Enter the hostname
34
type: string
35
required: true
36
port:
37
description: Enter the port
38
type: integer
39
default: 80
40
required: true
41
username:
42
description: Enter the username
43
type: string
44
required: true
45
46
triggers:
47
emit_greeting:
48
title: Trigger a New Greeting
49
description: Triggers a greeting every interval
50
input:
51
interval:
52
description: How frequently (in seconds) to trigger a greeting
53
type: integer
54
default: 15
55
required: true
56
output:
57
greeting:
58
required: true
59
type: person
60
61
actions:
62
say_goodbye:
63
title: Say Goodbye
64
description: Emit say goodbye message
65
input:
66
name:
67
type: string
68
description: Name to say goodbye to
69
required: true
70
output:
71
message:
72
title: Message
73
type: string
74
required: true

Let's review this file in detail.

Metadata Section

At the top of the spec, you define metadata about your plugin such as the name, version, and tags. The metadata section ends with two newline characters.

The table below lists all the metadata properties.

PropertyTypeDescription
plugin_spec_versionstringPlugin specification version which changes how the plugin operates
extensionstringExtension type, value should be: plugin
productsstringCompatible Insight products, value should be: [insightconnect]
namestringPlugin name for development purposes
titlestringPlugin name displayed in the UI
descriptionstringPlugin description displayed in the UI
versionstringPlugin version in semver format
vendorstringVendor who developed plugin, used for organization and The Extension Library
tagsarrayPlugin tags used as search terms in the UI and The Extension Library
hub_tagsobjectObject containing keys used for categorization in The Extension Library
resourcesobjectObject containing keys for linking in The Extension Library
supportstringDetermine ownership of plugin maintenance, values: rapid7, partner, community
statusarrayPlugin status to determine the state of the integration
enable_cachebooleanEnable storage on host for plugin to keep state

Below, you can find more details about each property.

plug_spec_version

This defines the plugin specification version, which changes how plugins operate.

  • v2 - is a standalone plugin as an HTTP REST server that is invoked once (you want this)
  • v1 - is a standalone plugin that exits after each invocation (legacy, and unsupported in insightConnect).

extension

This defines the type of extension. For plugins, it will be:

extension: plugin

More extension types may be available in the future.

products

This field defines compatible Insight products for categorization in The Extension Library.

For plugins, it will be:

products: [insightconnect]

name

The name of the plugin for development purposes. This value determines the directory and file system naming of the plugin.

E.g. name: rapid7_metasploit

See the style guide on how to properly set the name.

title

The title of the plugin is displayed in the InsightConnect UI. Users of the product see this value. The title should conform to proper noun rules.

E.g. title: Rapid7 Metasploit

Plugin title

description

The InsightConnect UI displays the plugin's description. Users of the product see this value. The description should conform to sentence-like structure.

E.g. description: Metasploit is the most widely used penetration testing framework

Plugin description

version

The plugin's version follows the semver format.

  • Plugins in development should begin with 0.1.0
  • Plugins in production should begin with 1.0.0

Plugin version

vendor

The vendor who developed the plugin, used for organization, searching, and display on The Extension Library

Plugin version

tags

Optional tags that describe your plugin's functionality. The InsightConnect UI and The Extension Library uses these tags to find your plugin.

Tags are created as an array of strings.

E.g. tags: [ malware, hash ]

hub_tags

Tags that describe your plugin's use cases, features, and searchable keywords for The Extension Library.

Tags are created as an array of strings.

yaml
1
hub_tags:
2
use_cases: [data_enrichment,threat_detection_and_response]
3
keywords: [malware, hash]
4
features: []

The following table lists valid use_cases values:

DisplaySpec Value
Data Enrichmentdata_enrichment
Alerting & Notificationsalerting_and_notifications
Application Managementapplication_management
Devopsdevops
Asset Inventoryasset_inventory
Container & Cloud Securitycloud_security
Credential Managementcredential_management
Data Utilitydata_utility
Offensive Securityoffensive_security
Remediation Managementremediation_management
Reporting & Analyticsreporting_and_analytics
Threat Dection & Responsethreat_detection_and_response
User Managementuser_management
Vulnerability Managementvulnerability_management

resources

Resources provides a place to link official URLs in The Extension Library.

An example for an open-source plugin on Github:

yaml
1
resources:
2
source_url: https://github.com/rapid7/insightconnect-plugins/tree/master/example
3
license_url: https://github.com/rapid7/insightconnect-plugins/blob/master/LICENSE
4
vendor_url: http://www.rapid7.com

If a key doesn't apply, then the key should be omitted in the spec.

support

Field to determine the ownership of the plugin for maintenance as well as to handle user requests.

Values can be: rapid7, partner, community.

E.g. support: community

status

Status describes the state of the integration as an array of strings. Therefore multiple statuses can be applied at the same time.

E.g. status: [deprecated]

Values are deprecated, insecure, and obsolete.

If none of these apply, leave the array empty:

E.g. status: []

enable_cache

Enables persistent storage on the host for a given plugin.

E.g. enable_cache: true

Base Types

The following types are available in the types, connection, trigger input/output, and action input/output sections.

The base type must be selected from one of the valid data types below:

TypeDescription
booleanTrue or false
integerInteger is a signed integer
floatFloating point number
stringString value
dateDate string value (In RFC 3339 format) per the JSON schema
bytesBytes are base64 encoded byte strings
objectGeneric JSON object
fileFile object with filename (string) and base64 content bytes
passwordHide typed strings for passwords and keys in the UI by masking the characters
pythonPython code
fileFile object consisting of filename of type string and content of type bytes
credential_username_passwordAuth object consisting of username of type string and password of type password
credential_asymmetric_keyAuth object consisting of key of type string for private keys
credential_secret_keyAuth object consisting of secretKey of type password for API keys
credential_tokenAuth object consisting of token of type password and optional domain of type string

You can also specify a collection (array) type by doing []<base type> wrapped in strings, e.g. type: "[]string". You must wrap it in double-quotes.

enum is an optional modifier that can be used to limit and declare the possible options for an input field. For example:

yaml
1
actions:
2
forward:
3
title: Forward Lookup
4
description: Forward DNS query
5
input:
6
domain:
7
type: string
8
description: Domain name to resolve
9
required: true
10
query:
11
type: string
12
description: Optional query type e.g. ANY, A, MX, NS, etc.
13
enum:
14
- A
15
- AAA
16
- ANY
17
- CNAME
18
- MX
19
- NS
20
- PTR
21
- SOA

The enumerated DNS query types are selectable from a drop-down box in the InsightConnect Workflow UI.

Enum

Custom Types Section

The types section defines any custom complex type objects you may need. It is in the format of a map which has the <type name> identifier as the key, and then an object definition that describes the property for the object. You can use these types later in your plugin specification as types for your input/output parameters, or within your connection definition.

The table below lists all of the types properties.

PropertyTypeDescription
<type name>identifierUnique identifier for developer, begins a new custom type
<identifier>identifierUnique identifier for developer, begins a new variable
titlestringTitle variable name displayed in InsightConnect UI
descriptionstringType variable description displayed in InsightConnect UI
typestringVariable data type
default<any>Default value for the variable
requiredbooleanWhether variable must be set to a value to continue
yaml
1
types:
2
<type name>:
3
<parameter no. 1 identifier>:
4
type: <choose from a valid type>
5
title: <optional, descriptive name for the UI>
6
description: <optional, string description>
7
default: <optional, default>
8
required: <optional, true or false, default is false. Set to true if required>
9
<parameter no. 2 identifier>:
10
type: <choose from a valid type>
11
title: <optional, descriptive name for the UI>
12
description: <optional, string description>
13
default: <optional, default>
14
required: <optional, true or false, default is false. Set to true if required>

In the example at the beginning of this page, we defined a custom type called person that has two properties: first_name and last_name.

These types will now be available in the connection sections, trigger input/output, and action input/output sections to use in addition to the base types.

yaml
1
...
2
output:
3
greeting:
4
title: Greeting
5
description: Greeting for person
6
required: true
7
type: person

Connection Section

The connection section defines the configuration variables you need for a connection. If your plugin does not require a connection, you can remove this section.

If a connection does exist, the UI will render a Configure Connection page for the values in the Workflow Builder.

In the connection section, each variable is defined in a map with its identifier. The identifier must be unique, including lowercase letters, numbers, or underscores only.

The table below lists all of the connection properties.

PropertyTypeDescription
<parameter>identifierUnique connection parameter identifier
titlestringConnection parameter name displayed in InsightConnect UI
descriptionstringConnection parameter description displayed in InsightConnect UI
typestringConnection parameter data type
requiredbooleanWhether parameter must be set to a value to continue
default<any>Default value for the connection parameter
examplestringA sample value that would be valid for this parameter
yaml
1
connection:
2
<parameter no. 1 identifier>:
3
title: <descriptive name for parameter>
4
description: <tooltip description string>
5
type: <choose from a valid type>
6
required: <optional, true or false, default is false. Set to true if required>
7
default: <optional, default>
8
example: <optional, sample value that would be valid>
9
<parameter no. 2 identifier>:
10
title: <descriptive name for parameter>
11
description: <tooltip description string>
12
type: <choose from a valid type>
13
required: <optional, true or false, default is false. Set to true if required>
14
default: <optional, default>
15
example: <optional, sample value that would be valid>

Input and output sections should not contain name identifiers.

Triggers Section

Triggers are long running processes that kick off workflows. All workflows must start with a trigger. This is in contrast to actions, which must occur after a trigger in a workflow. These actions are short lived processes that run, do something, and then die.

The code for triggers run in the body of a continuous loop, where the purpose is to poll for new events. When this event is found, the plugin triggers a workflow with the found event as JSON passes it to the next plugin in the workflow.

Example:

  1. Poll an RSS feed for new content
  2. When new content is available, pass the content as JSON to the engine
  3. Sleep for x seconds
  4. Repeat

A plugin can define 0 or more triggers. Simply create a section triggers and add a map (defined by a unique key) for each of the triggers you support.

The table below is a list of all the trigger properties.

PropertyTypeDescription
<identifier>identifierUnique trigger identifier for developer, begins a new trigger's section
titlestringTrigger name displayed in InsightConnect UI
descriptionstringTrigger description displayed in InsightConnect UI
inputmapSection to define supported input variables from the user
outputmapSection to define supported output variables from the user
yaml
1
triggers:
2
<unique trigger identifier>:
3
title: <optional, descriptive name for the UI>
4
description: <optional, string description>
5
input: <optional, map of identifier -> variable inputs>
6
output: <map of identifier -> variable outputs>

Input and output sections should not contain name identifiers. For usability, all triggers should support an input called frequency, which is passed to a sleep mechanism in the SDK.

yaml
1
frequency:
2
type: integer
3
description: Poll frequency in seconds
4
default: 300
5
required: false

Actions Section

Each plugin may contain zero or more actions, which are short-running processes that execute, return results, and then exit. Workflows typically contain many actions chained together. Plugin code is executed in a Docker container. As the action initializes, the connection object is created and then the action code runs.

The table below lists all of the actions properties.

yaml
1
actions:
2
<unique action identifier>:
3
title: <optional, descriptive name for the UI>
4
description: <optional, string description>
5
input: <optional, map of identifier -> variable inputs>
6
output: <map of identifier -> variable outputs>

UI Rendering

The UI renders the name, title, description, etc. properties from the spec file in the following ways.

Action & Trigger Selection

Action Selection

Action & Trigger Configuration

Action Configuration

Creating a Plugin

Now that you have a valid plugin spec file, you can codegenerate a plugin shell. You need to have the icon-plugin tool. The latest tooling is available here.

Run it as follows:

icon-plugin generate python plugin.spec.yaml

You may change plugin.spec.yaml to the path of your plugin spec. By default, it will generate the plugin folder with the value of name defined in your plugin spec yaml. You can also use the --path flag to specify a custom path to generate your plugin inside of e.g. --path /tmp/example.

Our plugin above was generated in /tmp/example. You should be able to build it now by going to the folder and typing make.

1
$ icon-plugin generate python example.yaml --path /tmp/example
2
INFO[0000] Writing to file: /tmp/example/komand_example/__init__.py
3
INFO[0000] Writing to file: /tmp/example/komand_example/util/__init__.py
4
INFO[0000] Writing to file: /tmp/example/Makefile
5
INFO[0000] Writing to file: /tmp/example/.gitignore
6
INFO[0000] Writing to file: /tmp/example/setup.py
7
INFO[0000] Writing to file: /tmp/example/Dockerfile
8
INFO[0000] Writing to file: /tmp/example/requirements.txt
9
INFO[0000] Writing to file: /tmp/example/bin/komand_example
10
INFO[0000] Writing to file: /tmp/example/komand_example/connection/connection.py
11
INFO[0000] Writing to file: /tmp/example/komand_example/connection/__init__.py
12
INFO[0000] Writing to file: /tmp/example/komand_example/connection/schema.py
13
INFO[0000] Writing to file: /tmp/example/komand_example/actions/say_goodbye/action.py
14
INFO[0000] Writing to file: /tmp/example/komand_example/actions/say_goodbye/schema.py
15
INFO[0000] Writing to file: /tmp/example/komand_example/actions/say_goodbye/__init__.py
16
INFO[0000] Writing to file: /tmp/example/komand_example/actions/__init__.py
17
INFO[0000] Writing to file: /tmp/example/komand_example/triggers/emit_greeting/trigger.py
18
INFO[0000] Writing to file: /tmp/example/komand_example/triggers/emit_greeting/schema.py
19
INFO[0000] Writing to file: /tmp/example/komand_example/triggers/emit_greeting/__init__.py
20
INFO[0000] Writing to file: /tmp/example/komand_example/triggers/__init__.py
21
INFO[0000] Code generation completed! Your plugin is located at: /tmp/example

The make task does two things: Creates a <vendor>/<name>:<version> container with the plugin code, and also a <vendor>-<name>-<version>.tar.gz package that can be uploaded via icon-plugin register or via the plugins UI within the InsightConnect interface.

All of the plugins are built as Docker containers for easy packaging and running.

You can always type make during the process of writing your plugin to build the latest version of it. Any code changes require packaging for use, so get in the habit of running make often.

You now have a plugin shell you can interact with. To continue, see the next page on Running Plugins.