Style Guide

This style guide covers the style for all plugin components, including the plugin's spec, code, dependencies, and documentation.

Conventions

The following sections document conventions that should be followed while writing plugins.

Where rules are not defined in this document, follow PEP8.

Quick Guide

• For dependencies, pin OS package and Python package versions.
• For security, set least privileged account with USER nobody in the Dockerfile when possible.
• For size, use the slim SDK images when possible: komand/python-3-37-slim-plugin and komand/python-3-37-plugin.
• For error handling, use PluginException and ConnectionTestException.
• For logging, use self.logger.
• For docs, use the changelog style.
• For docs, validate markdown with make validate which calls mdl to lint help.md.

Plugin Names

Regarding the name of the plugin as defined in the plugin.spec.yaml file. Note that this should be the only key called name in the entire plugin spec (all others should be renamed to title.)

name: plugin_name

Rules

• Names should be lowercase e.g. myplugin
• Use the company_product format where possible
  • e.g. rapid7_metasploit
• Names should be succinct and represent their purpose or servicename
  • e.g. Service-Now should be servicenow
• Use underscores to separate words if not a company or service name
  • e.g. get_url
• Where the service and website are the same, if the domain is unique enough, avoid top-level domain
  • e.g. freegeoip.net should be freegeoip
• Where the service and website are the same, if the domain is not unique, add top-level domain
  • e.g. ifconfig is unixtool, web service ifconfig.co should be ifconfig_co
• Numbers are valid in plugin names
  • e.g. geolite2
• Characters other than alpha-numeric and underscore are not allowed
• Names should be four words or less

Spec

Style information for the plugin.spec.yaml file.

Indentation

Use two spaces for indentation:

yaml
1
triggers:
2
  my_trigger1:
3
    blah:
4
      blah:
5
  my_trigger2:
6
    blah:
7
      blah:

Line Breaks

Schema sections: metadata, triggers, action, connections, should be separated by a line break. In addition, there should not be extra newlines at the end of file.

Example:

yaml
1
...
2
tags: [ "blah" ]
3

4
triggers:
5
  my_trigger1:
6
    blah:
7
      blah:
8
  my_trigger2:
9
    blah:
10
      blah:
11

12
  actions:
13
    my_action1:
14
      blah:
15
        blah:
16
    my_action2:
17
      blah:
18
        blah:

Quoting

Of all the types, only the array types need to be double-quoted.

Good:

yaml
1
input:
2
  array:
3
    description: Array of things
4
    type: "[]string"

Bad:

yaml
1
input:
2
  url:
3
    type: "string" # This shouldn't be quoted
4
    description: URL to Download
5
    required: "true" # This shouldn't be quoted

Punctuation

Do not add end of sentence punctuation to titles and descriptions. They should not be present in any other fields either. Punctuation should be used in the body of the help section of the spec.

Tags

Requirements: * Lower case characters * Numbers permitted * No underscores

yaml
1
tags: [ malware, ioc, rat, trojan, remote ]

Commit Messages

Git commits should be descriptive and describe the fix or change.

Commit Message Rules

• Use the imperative
  • e.g. Fix instead of Fixed or Fixes
  • First letter of description should be capitalized
  • Use form: <file>: <description> where possible

See Git Best Practices

Action & Trigger Names

Names are specified in the plugin.spec.yaml file using the title key. Example of good style is below:

yaml
1
actions:
2
  match_string:
3
    title: Match String
4
    description: Match string from a text file
5
  match_number:
6
    title: Match Number
7
    description: Match number from a text file
8
  extract:
9
    title: Extract
10
    description: Extract files from archive

Action & Trigger Naming Rules

• title shouldn't be any more than four words. The description is intended to give the rest of the detail.
• Each word in the title should have the first letter capitalized (e.g. Match String since it's a title.) The exception to this rule are articles and conjunctions (e.g. Parse an Assortment).
• First word in the description should have the first letter capitalized and not be same as title (e.g. Match string from text file).

Property Names

Property names are defined here. These include input/output variables, actions, triggers, and connections names.

Common

• host - Neutral: Used when either an IP address or domain name can be the value.
• address - Used for IP address values only (IPv4 or IPv6) e.g. 8.8.8.8
• domain - Used for domain name values only e.g. www.google.com
• url - Used for url values only e.g. https://product.company.com:1234
• ssl_verify - Boolean property used to decide whether to verify a SSL certificate

Property Naming Rules

• Names should be lowercase e.g. timeout
• Names should be succinct and represent their purpose, limit of 2 words if possible e.g. country_code
• Use underscores to separate words if not succinct metro_code
• Characters other than alpha and underscores are not allowed unless required by API

Input Examples

Input examples need to be added for each input of action, trigger, and connection in plugin.spec.yaml and help.md. This will help users figure out the input value's format.

The table below lists few example types and values.

Changelog

We document changelog history in the Version section of the plugin’s help property located in plugin.spec.yaml. The format is x.x.x - <Message> where the x’s represent the semantic version and a message documents the change.

Below is a list of message conventions to use for common changes to plugins.

• Support web server mode - For web server mode support i.e. New parent Docker image, v2 plugin spec version, and new self.logger use
• Update to new credential types - To support new credential types e.g. credential_username_password, credential_secret_key
• Update to v2 Python plugin architecture - For porting to Python v2 arch
• Update to new secret key credential type - For use of a single new credential type, specific to credential_secret_key in this example
• New action <Action Title> - For a new action e.g. New action Remove from Policy
• New actions <Action Title> and <Action Title> - For a list of two new actions
• New actions <Action Title>, <Action Title> and <Action Title> - For a list of more than two new actions
• New trigger <Trigger Title> - For new trigger
• New triggers <Trigger Title> and <Trigger Title> - For a list of two new triggers
• New actions <Action Title>, <Action Title> and <Action Title> - For a list of more than two new triggers
• New trigger <Trigger Title> | New actions <Action Title>, <Action Title> and <Action Title> - For a new trigger and multiple actions
• Fix issue where <bug description> - For bug fixes e.g. Fix issue in connection that can cause a failure with valid credentials
• Update to use the komand/python-3-slim-plugin:2`` Docker image to reduce plugin size - For using then new SDK images
• Run plugin as least privileged user - To indicate setting of USER nobody in the Dockerfile

For many at once, concatenate them with a | character, e.g.:

• Update to v2 Python plugin architecture | Support web server mode | Update to new credential types
• Update to new credential types | New action Disable User

Errors

See the Error Handling in Integrations guide.

Triggers

Inputs

For usability, all triggers should support an input called interval that is passed to the sleep mechanism in the SDKs language:

yaml
1
frequency:
2
  type: integer
3
  description: Poll frequency in seconds
4
  default: 300
5
  required: false

Connections

Style

Connections follow the same style guide as Property Names.

Logging

Plugins that do not use connections should remove the default self.logger.info statement from the connection.py body and add a pass statement.

python
1
def connect(self, params):
2
    """
3
    Connection config params are supplied as a dict in
4
    params or also accessible in self.parameters['key']
5

6
    The following will setup the var to be accessed
7
      self.blah = self.parameters['blah']
8
    in the action and trigger files as:
9
      blah = self.connection.blah
10
    """
11
    # TODO: Implement connection or 'pass' if no connection is necessary
12
    self.logger.info("Connect: Connecting...")

Here's what that looks like when followed:

python
1
def connect(self, params):
2
    """
3
    Connection config params are supplied as a dict in
4
    params or also accessible in self.parameters['key']
5

6
    The following will setup the var to be accessed
7
      self.blah = self.parameters['blah']
8
    in the action and trigger files as:
9
      blah = self.connection.blah
10
    """
11
    pass

TODO

Remove the TODO lines that are automatically generated with the plugin, once all tasks have been completed.

Notice the # TODO lines from the newly generate action below. These lines should be removed when code is added.

python
1
import komand
2
from .schema import DomainLookupInput, DomainLookupOutput
3
# Custom imports below
4

5

6
class DomainLookup(insightconnect_plugin_runtime.Action):
7

8
    def __init__(self):
9
        super(self.__class__, self).__init__(
10
                name='domain_lookup',
11
                description='Lookup Domain Name',
12
                input=DomainLookupInput(),
13
                output=DomainLookupOutput())
14

15
    def run(self, params={}):
16
        # TODO: Implement run function
17
        return {}
18

19
    def test(self):
20
        # TODO: Implement test function
21
        return {}

Notice the # TODO lines from the newly generated connection.py file below. This line should be removed.

python
1
import komand
2
from .schema import ConnectionSchema
3
# Custom imports below
4

5

6
class Connection(insightconnect_plugin_runtime.Connection):
7

8
    def __init__(self):
9
        super(self.__class__, self).__init__(input=ConnectionSchema())
10

11
    def connect(self, params):
12
        """
13
        Connection config params are supplied as a dict in
14
        params or also accessible in self.parameters['key']
15

16
        The following will setup the var to be accessed
17
          self.blah = self.parameters['blah']
18
        in the action and trigger files as:
19
          blah = self.connection.blah
20
        """
21
        # TODO: Implement connection or 'pass' if no connection is necessary
22
        self.logger.info("Connect: Connecting...")

Tests

Each plugin should include JSON test files in the tests directory. These test files are used to simulate user input to the plugin to test its various actions and triggers.

The ./run.sh tool in each plugin’s directory can be used to list the available samples and generate them.

List samples

 
1
$ ./run.sh -c sample
2
Actions: [domain_lookup domain_blacklist address_blacklist url_lookup address_lookup]
3
Triggers: [poll_domain_blacklist poll_address_blacklist]
4
Sample requires sample name e.g. ``./run.sh -c sample <name>''

Generate a sample for the domain_lookup action and write it to the tests directory.

$ ./run.sh -c sample domain_lookup > tests/domain_lookup.json

Generate all samples and write them to the tests directory.

$ ./run.sh -c samples

In the event where the ./run.sh is not available, perform manual generation with:

docker run komand/<plugin>:<version> sample <action> | jq '.' > tests/<testname>.json

jq is used to pretty print the files which makes them easier to edit.

Testing Rules

• Success tests for all actions
• Failure tests for all actions
• Tests for all optional inputs and for all actions
• Failure tests should be suffixed with _bad e.g. png_download_bad.json
• Success tests should not be suffixed e.g. png_download.json

Examples

• Success: <desc>_<action>.json e.g. domain_lookup.json, github_lookup.json
• Failure: <desc>_<action>_bad.json e.g. nxdomain_lookup_bad.json, private_ip_lookup_bad.json

Examples from the csv plugin:

 
1
tests:
2
├── 1r_filter.json
3
├── 1r_filter_bad.json
4
├── 1r_filter_bad2.json
5
├── 1r_filter_bad3.json
6
├── 1r_filter_bad4.json
7
├── 1r_space_filter.json
8
└── csv_filter_bad.json

Dependencies

Dockerfile

All plugins can use and modify the Dockerfile to add dependencies to the plugin. Dependencies are typically added with the RUN and/or ADD Dockerfile commands.

In the following example, we need a Debian package, a custom python library not available in Python repositories, and a configuration file so the plugin may run successfully:

sh
1
FROM komand/komand/python-3-37-slim-plugin
2
# Refer to the following documentation for available SDK parent images: https://komand.github.io/python/sdk.html#version
3

4
LABEL organization=komand
5
LABEL sdk=python
6
LABEL type=plugin
7

8
# Add any custom package dependencies here
9
# NOTE: Add pip packages to requirements.txt
10
RUN apt-get update && apt-get install -y whois=5.2.7
11
RUN pip install git+https://github.com/komand/python-whois.git@0.4.0
12

13
# End package dependencies
14

15
# Add configuration file from plugin repo
16
ADD ./whois.conf /etc/whois.conf
17

18
# Install pip dependencies
19
RUN if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
20

21
# Install plugin
22
RUN python setup.py build && python setup.py install
23

24
# User to run plugin code. The two supported users are: root, nobody
25
USER nobody
26

27
ENTRYPOINT ["/usr/local/bin/komand_whois"]

Notice that pip is installed by default in the Docker image but Python packages should not be installed using this method unless they cannot be found using the methods described below. The example above was a case where it was not available using the other methods.

Rules:

• Pinned versions of a dependency package are required
• For Debian packages, apt-get update must be used before package installation
• Should not be used for Python package installation unless required by technical limitations

requirements.txt

Newer plugins (v2 arch) use the requirements.txt file to obtain dependencies. If a plugin does not contain this file, then the older setup.py file is used.

Rules:

• Pinned versions of a dependency package are required in requirements.txt
• Do not touch setup.py

E.g.:

sh
1
$ cat powershell/requirements.txt
2
# List third-party dependencies here, separated by newlines.
3
# All dependencies must be version-pinned, eg. requests==1.2.0
4
# See: https://pip.pypa.io/en/stable/user_guide/#requirements-files
5
pywinrm==0.2.2
6
pykerberos==1.2.1
7
requests-kerberos==0.12.0

setup.py

Older pllugins (V1 arch) use the setup.py file to obtain dependencies. If a plugin contains requirements.txt then this file should not be modified and see the above section.

Rules:

• Pinned versions of a dependency package are required in setup.py's install_requires section

E.g.

sh
1
$ cat zeus_tracker/setup.py
2
from setuptools import setup, find_packages
3

4
setup(name='zeus_tracker-komand-plugin',
5
      version='0.1.1',
6
      description='ZeuS Command&Control Server Tracking',
7
      author='komand',
8
      author_email='',
9
      url='',
10
      packages=find_packages(),
11
      install_requires=['komand', 'feedparser==5.2.1', 'lxml==4.1.1'],
12
      scripts=['bin/komand_zeus_tracker']
13
      )