Style Guide

This style guide covers the style for all plugin components including the plugin's spec, code, dependencies, and documentation.

Conventions

The following sections document conventions that should be followed while writing plugins.

Where rules are not defined in this document, follow PEP8.

Quick Guide

Plugin Names

Regarding the name of the plugin as defined in the plugin.spec.yaml file. Note that this should be the only key called name in the entire plugin spec (all others should be renamed to title.)

name: plugin_name

Rules

  • Names should be lowercase e.g. myplugin
  • Use the company_product format where possible
    • e.g. rapid7_metasploit
  • Names should be succinct and represent their purpose or servicename
    • e.g. Service-Now should be servicenow
  • Use underscores to separate words if not a company or service name
    • e.g. get_url
  • Where the service and website are the same, if the domain is unique enough, avoid top-level domain
    • e.g. freegeoip.net should be freegeoip
  • Where the service and website are the same, if the domain is not unique, add top-level domain
    • e.g. ifconfig is unixtool, web service ifconfig.co should be ifconfig_co
  • Numbers are valid in plugin names
    • e.g. geolite2
  • Characters other than alpha-numeric and underscore are not allowed
  • Names should be four words or less

Spec

Style information for the plugin.spec.yaml file.

Indentation

Use two spaces for indentation:

yaml
1
triggers:
2
my_trigger1:
3
blah:
4
blah:
5
my_trigger2:
6
blah:
7
blah:

Line Breaks

Schema sections: metadata, triggers, action, connections, should be separated by a line break. In addition, there should not be extra newlines at the end of file.

Example:

yaml
1
...
2
tags: [ "blah" ]
3
4
triggers:
5
my_trigger1:
6
blah:
7
blah:
8
my_trigger2:
9
blah:
10
blah:
11
12
actions:
13
my_action1:
14
blah:
15
blah:
16
my_action2:
17
blah:
18
blah:

Quoting

Of all the types, only the array types need to be double-quoted.

Good:

yaml
1
input:
2
array:
3
description: Array of things
4
type: "[]string"

Bad:

yaml
1
input:
2
url:
3
type: "string" # This shouldn't be quoted
4
description: URL to Download
5
required: "true" # This shouldn't be quoted

Punctuation

Do not add end of sentence punctuation to titles and descriptions. They should not be present in any other fields either. Punctuation should be used in the body of the help section of the spec.

Tags

Requirements: * Lower case characters * Numbers permitted * No underscores

yaml
1
tags: [ malware, ioc, rat, trojan, remote ]

Commit Messages

Git commits should be descriptive and describe the fix or change.

Rules

  • Use the imperative

    • e.g. Fix instead of Fixed or Fixes
    • First letter of description should be capitalized
    • Use form: <file>: <description> where possible

    See Git Best Practices

Action & Trigger Names

Names are specified in the plugin.spec.yaml file using the title key. Example of good style is below:

yaml
1
actions:
2
match_string:
3
title: Match String
4
description: Match string from a text file
5
match_number:
6
title: Match Number
7
description: Match number from a text file
8
extract:
9
title: Extract
10
description: Extract files from archive

Rules

  • title shouldn't be any more than four words. The description is intended to give the rest of the detail.
  • Each word in the title should have the first letter capitalized (e.g. Match String since it's a title.) The exception to this rule are articles and conjunctions (e.g. Parse an Assortment).
  • First word in the description should have the first letter capitalized and not be same as title (e.g. Match string from text file).

Property Names

Property names are defined here. These include input/output variables, actions, triggers, and connections names.

Common

  • host - Neutral: Used when either an IP address or domain name can be the value.
  • address - Used for IP address values only (IPv4 or IPv6) e.g. 8.8.8.8
  • domain - Used for domain name values only e.g. www.google.com
  • url - Used for url values only e.g. https://product.company.com:1234
  • ssl_verify - Boolean property used to decide whether to verify a SSL certificate

Rules

  • Names should be lowercase e.g. timeout
  • Names should be succinct and represent their purpose, limit of 2 words if possible e.g. country_code
  • Use underscores to separate words if not succinct metro_code
  • Characters other than alpha and underscores are not allowed unless required by API

Input Examples

Input examples need to be added for each input of action, trigger, and connection in plugin.spec.yaml and help.md. This will help users figure out the input value's format.

The table below lists few example types and values.

Example typeExample value
CompanyExample Organization
Domainsexample.com
rapid7.com
URLhttps://example.com
https://rapid7.com
IP AddressThe blocks 192.0.2.0/24 (TEST-NET-1), 198.51.100.0/24 (TEST-NET-2), and 203.0.113.0/24 (TEST-NET-3) are provided for use in documentation. RFC5737
IP Gateway198.51.100.1
IP Broadcast198.51.100.255
IP CIDR198.51.100.0/24
IPv6 Address2001:db8:1:1:1:1:1:1 (RFC3849 - addresses reserved for documentation)
Emailuser@example.com
Array: ["user1@example.com", "user2@example.com"]
To differentiate domains: user@rapid7.com, and user2@rapid7.com
Usernameuser1
FullnameExample User
Phone NumberUS number in the range (800) 555-0100 through (800) 555-0199. That range is reserved for use in examples and in fiction.
General MD5 Hash9de5069c5afe602b2ea0a04b66beb2c0
General SHA1 Hash02699626f388ed830012e5b787640e71c56d42d8
Bytes FileUmFwaWQ3IEluc2lnaHRDb25uZWN0Cg==
Malware filenamesetup.exe
credential_username_password{ "username": "user1", "password": "mypassword" } - if password is not a password, like Jira API key - make sure your format a realistic but sanitized key
credential_secret_key{ "domain": "example.com", "token": "9de5069c5afe602b2ea0a04b66beb2c0" } - Update token to match vendor’s key format as needed. If key is MD5 hash (32 chars w/o newline) then use content of echo "Rapid7 InsightConnect" | md5
credential_asymmetric_key-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAjGnoUtfPHqvX3PIU6N9FKmwQ3Zl+NoaWb4yMLhudkdEBJ3Au
+I8QdlqDKBm656UeOCh3r/i9e0ULKxkXDFfKmc3p2Wv+0lVOYGvxZFKUwKH0riAL
A4imyYuL/fweOSGSnQlgYKr99HciTBIdL15SZ32TjYb+PDZBl+6zQsw2HYNJcqMj
iciC7CAj6gB9SO8x1vMsRkU+rqKuc2r8Uk+qhECw8zR4K66wFuYM17sGUMXUq/pH
WdiEvO3q/mdK47Nrx5i2baC7o5RXspKHYy6Xer4Vbnipl4DgAKkaNOL02a+Zv38Q
l+xy9wdmWqUIbMiqSbj/k6xxDiPQkTR+/032eQIDAQABAoIBAEkPzpBUtPQbrJ3N
5S1rB71UL85u0OqkS2DNvB89xVabb0NLL1Wsc39yB271PHjORRQpkmWhQ08CFRae
3oxQnh47s+OrOxPMyZSIdjmicr5tRzjXeYOkNk0G7JgC+OL3YieOOnTyZGQxHUqB
3mfIZ45sHDv3MxC3lpfs35/xTHM8E/gW/gTfvU3QboQaL1q/taRQYEHvgiutwdZ0
sEFtJ8eAwOBABXiV3QPxnAQgIpwYpbicl3AK15gs5ENK4Rngi2bI7hdmMwDWa6t/
g0CP0TityFq05JUmnaz4wekXxD5EBm776EYNSoxTCaSzTMYwZCITrqXl6Y4/ogeT
uVSm9ZECgYEA7G8CyyDKDTBYoIyEknJVKSwuelOA2edxmVyKL8hLoPiq1QoSH/N1
30nN/GVcvD7QED4p/u0XaMuPm2HVhuXwxu/t9j11DVlKP7QsH9u4pJKziw6NmV5N
/9+mcjdWAH5BqaJtmpF0uoZsWk41JVe0fA7a3FCrXp1U/GD9BKSAD00CgYEAmAio
ChEh7+pD7vutF85u+FqbdjY+KmyFeTPd2717P6i5V6C6lVpcnM7voZlGy0fjoald
e9ntm0VU8FZkUIihKPzW9/LSAV8BgO+vSQrN/IMEmDqol959IxxI/6yzkY5JwYRP
mlwoNzU0ekcHzg0eu7DA1uzRfv4F1NUW+QylRd0CgYEAzr07OhdP1jyCItD8U3n6
EWh6s6g0sVV5tdp/UszXpMgLyQFnW9ztIvRMU/jmIAzkrm9NFYaHw7DLv9jKd4y0
/59o+ro+kg+TpySKuMjOKcnFiUCOfJ9DoQwVZSYR45iDHivTnya1ZSyJrmVYf3Cz
dw8ePSukzbTRTWYZmGenOrkCgYEAhoO6MdYAweXzH0J8XsDePEzmmcvaauzDl35F
gIOAxc1B1381NqnRoUgSi1czZO6BP+q69LbX3PaV9WNqtDp+5OX4ST8FggMOMIdg
/m5Z3F4LtajIvD41V9hR2i1yX4mWRmsLh1acmmQvvzSTekLvez8jD8ZOgV69yBaV
kdsXa90CgYEAk+6ghpXNku12UANf9MH8loN+35/iPeeoqf0MY5FMVRYx10ZA91Lh
ieAczVhiqzxCtHWhLA4SxE962eg+ji/awkS4kXLCMuZIESE+jFc7ptUmJjlsOWjv
8/dqUH5yjRKs2qxkBWG4HmT3Nx6A8sYIrUYxyqVLBpG8yKngbnaYPV4=
-----END RSA PRIVATE KEY-----
File{ "filename": "<name>", "content": "UmFwaWQ3IEluc2lnaHRDb25uZWN0Cg==" } We can use a custom filename fitting for the purpose of the action. E.g. if action submits a PDF, CompanyData.pdf. (Content is: echo "Rapid7 InsightConnect" | base64)
Hash type CRC326851cf3c
Hash type MD544d88612fea8a8f36de82e1278abb02f
Hash type SHA13395856ce81f2b7382dee72602f798b642f14140
Hash type SHA224b42ec8b47deb2dc75edebd01132d63f8e8d4cd08e5d26d8bd366bdc5
Hash type SHA256275a021bbfb6489e54d471899f7db9d1663fc695ec2fe2a2c4538aabf651fd0f
Hash type SHA384038f2e50e33dacef50d7e503b45c3525fcdbe89a823f9c4417d7c13e8e96a53dd6bd6d7fcc91189c5cda7253f4455106
Hash type SHA512cc805d5fab1fd71a4ab352a9c533e65fb2d5b885518f4e565e68847223b8e6b85cb48f3afad842726d99239c9e36505c64b0dc9a061d9e507d833277ada336ab

Changelog

We document changelog history in the Version section of the plugin’s help property located in plugin.spec.yaml. The format is x.x.x - <Message> where the x’s represent the semantic version and a message documents the change.

Below is a list of message conventions to use for common changes to plugins.

  • Support web server mode - For web server mode support i.e. New parent Docker image, v2 plugin spec version, and new self.logger use
  • Update to new credential types - To support new credential types e.g. credential_username_password, credential_secret_key
  • Update to v2 Python plugin architecture - For porting to Python v2 arch
  • Update to new secret key credential type - For use of a single new credential type, specific to credential_secret_key in this example
  • New action <Action Title> - For a new action e.g. New action Remove from Policy
  • New actions <Action Title> and <Action Title> - For a list of two new actions
  • New actions <Action Title>, <Action Title> and <Action Title> - For a list of more than two new actions
  • New trigger <Trigger Title> - For new trigger
  • New triggers <Trigger Title> and <Trigger Title> - For a list of two new triggers
  • New actions <Action Title>, <Action Title> and <Action Title> - For a list of more than two new triggers
  • New trigger <Trigger Title> | New actions <Action Title>, <Action Title> and <Action Title> - For a new trigger and multiple actions
  • Fix issue where <bug description> - For bug fixes e.g. Fix issue in connection that can cause a failure with valid credentials
  • Update to use the komand/python-3-slim-plugin:2`` Docker image to reduce plugin size - For using then new SDK images
  • Run plugin as least privileged user - To indicate setting of USER nobody in the Dockerfile

For many at once, concatenate them with a | character, e.g.:

  • Update to v2 Python plugin architecture | Support web server mode | Update to new credential types
  • Update to new credential types | New action Disable User

Errors

See the Error Handling in Integrations guide.

Triggers

Inputs

For usability, all triggers should support an input called interval that is passed to the sleep mechanism in the SDKs language:

yaml
1
frequency:
2
type: integer
3
description: Poll frequency in seconds
4
default: 300
5
required: false

Connections

Style

Connections follow the same style guide as Property Names.

Logging

Plugins that do not use connections should remove the default self.logger.info statement from the connection.py body and add a pass statement.

python
1
def connect(self, params):
2
"""
3
Connection config params are supplied as a dict in
4
params or also accessible in self.parameters['key']
5
6
The following will setup the var to be accessed
7
self.blah = self.parameters['blah']
8
in the action and trigger files as:
9
blah = self.connection.blah
10
"""
11
# TODO: Implement connection or 'pass' if no connection is necessary
12
self.logger.info("Connect: Connecting...")

Here's what that looks like when followed:

python
1
def connect(self, params):
2
"""
3
Connection config params are supplied as a dict in
4
params or also accessible in self.parameters['key']
5
6
The following will setup the var to be accessed
7
self.blah = self.parameters['blah']
8
in the action and trigger files as:
9
blah = self.connection.blah
10
"""
11
pass

TODO

Remove the TODO lines that are automatically generated with the plugin, once all tasks have been completed.

Notice the # TODO lines from the newly generate action below. These lines should be removed when code is added.

python
1
import komand
2
from .schema import DomainLookupInput, DomainLookupOutput
3
# Custom imports below
4
5
6
class DomainLookup(komand.Action):
7
8
def __init__(self):
9
super(self.__class__, self).__init__(
10
name='domain_lookup',
11
description='Lookup Domain Name',
12
input=DomainLookupInput(),
13
output=DomainLookupOutput())
14
15
def run(self, params={}):
16
# TODO: Implement run function
17
return {}
18
19
def test(self):
20
# TODO: Implement test function
21
return {}

Notice the # TODO lines from the newly generated connection.py file below. This line should be removed.

python
1
import komand
2
from .schema import ConnectionSchema
3
# Custom imports below
4
5
6
class Connection(komand.Connection):
7
8
def __init__(self):
9
super(self.__class__, self).__init__(input=ConnectionSchema())
10
11
def connect(self, params):
12
"""
13
Connection config params are supplied as a dict in
14
params or also accessible in self.parameters['key']
15
16
The following will setup the var to be accessed
17
self.blah = self.parameters['blah']
18
in the action and trigger files as:
19
blah = self.connection.blah
20
"""
21
# TODO: Implement connection or 'pass' if no connection is necessary
22
self.logger.info("Connect: Connecting...")

Tests

Each plugin should include JSON test files in the tests directory. These test files are used to simulate user input to the plugin to test its various actions and triggers.

The ./run.sh tool in each plugin’s directory can be used to list the available samples and generate them.

List samples

1
$ ./run.sh -c sample
2
Actions: [domain_lookup domain_blacklist address_blacklist url_lookup address_lookup]
3
Triggers: [poll_domain_blacklist poll_address_blacklist]
4
Sample requires sample name e.g. ``./run.sh -c sample <name>''

Generate a sample for the domain_lookup action and write it to the tests directory.

$ ./run.sh -c sample domain_lookup > tests/domain_lookup.json

Generate all samples and write them to the tests directory.

$ ./run.sh -c samples

In the event where the ./run.sh is not available, perform manual generation with:

docker run komand/<plugin>:<version> sample <action> | jq '.' > tests/<testname>.json

jq is used to pretty print the files which makes them easier to edit.

Rules

  • Success tests for all actions
  • Failure tests for all actions
  • Tests for all optional inputs and for all actions
  • Failure tests should be suffixed with _bad e.g. png_download_bad.json
  • Success tests should not be suffixed e.g. png_download.json

Examples

  • Success: <desc>_<action>.json e.g. domain_lookup.json, github_lookup.json
  • Failure: <desc>_<action>_bad.json e.g. nxdomain_lookup_bad.json, private_ip_lookup_bad.json

Examples from the csv plugin:

1
tests:
2
├── 1r_filter.json
3
├── 1r_filter_bad.json
4
├── 1r_filter_bad2.json
5
├── 1r_filter_bad3.json
6
├── 1r_filter_bad4.json
7
├── 1r_space_filter.json
8
└── csv_filter_bad.json

Dependencies

Dockerfile

All plugins can use and modify the Dockerfile to add dependencies to the plugin. Dependencies are typically added with the RUN and/or ADD Dockerfile commands.

In the following example, we need a Debian package, a custom python library not available in Python repositories, and a configuration file so the plugin may run successfully:

1
FROM komand/komand/python-3-37-slim-plugin
2
# Refer to the following documentation for available SDK parent images: https://komand.github.io/python/sdk.html#version
3
4
LABEL organization=komand
5
LABEL sdk=python
6
LABEL type=plugin
7
8
# Add any custom package dependencies here
9
# NOTE: Add pip packages to requirements.txt
10
RUN apt-get update && apt-get install -y whois=5.2.7
11
RUN pip install git+https://github.com/komand/python-whois.git@0.4.0
12
13
# End package dependencies
14
15
# Add configuration file from plugin repo
16
ADD ./whois.conf /etc/whois.conf
17
18
# Install pip dependencies
19
RUN if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
20
21
# Install plugin
22
RUN python setup.py build && python setup.py install
23
24
# User to run plugin code. The two supported users are: root, nobody
25
USER nobody
26
27
ENTRYPOINT ["/usr/local/bin/komand_whois"]

Notice that pip is installed by default in the Docker image but Python packages should not be installed using this method unless they cannot be found using the methods described below. The example above was a case where it was not available using the other methods.

Rules:

  • Pinned versions of a dependency package are required
  • For Debian packages, apt-get update must be used before package installation
  • Should not be used for Python package installation unless required by technical limitations

requirements.txt

Newer plugins (v2 arch) use the requirements.txt file to obtain dependencies. If a plugin does not contain this file, then the older setup.py file is used.

Rules:

  • Pinned versions of a dependency package are required in requirements.txt
  • Do not touch setup.py

E.g.

1
$ cat powershell/requirements.txt
2
# List third-party dependencies here, separated by newlines.
3
# All dependencies must be version-pinned, eg. requests==1.2.0
4
# See: https://pip.pypa.io/en/stable/user_guide/#requirements-files
5
pywinrm==0.2.2
6
pykerberos==1.2.1
7
requests-kerberos==0.12.0

setup.py

Older pllugins (V1 arch) use the setup.py file to obtain dependencies. If a plugin contains requirements.txt then this file should not be modified and see the above section.

Rules:

  • Pinned versions of a dependency package are required in setup.py's install_requires section

E.g.

1
$ cat zeus_tracker/setup.py
2
from setuptools import setup, find_packages
3
4
setup(name='zeus_tracker-komand-plugin',
5
version='0.1.1',
6
description='ZeuS Command&Control Server Tracking',
7
author='komand',
8
author_email='',
9
url='',
10
packages=find_packages(),
11
install_requires=['komand', 'feedparser==5.2.1', 'lxml==4.1.1'],
12
scripts=['bin/komand_zeus_tracker']
13
)