Regenerating Plugins

Regeneration occurs when you want to update the plugin schema to add, remove, or modify any actions, triggers, improved title and descriptions, etc. without loss of code and manual updates to the JSON schema.

Compatability

The latest plugin architecture is V2, which refers to SDK and icon-plugin tool improvements. The improvements split plugin components such as action and triggers into their own directories. For each directory, a code (e.g. action.py) and schema (e.g. schema.py) file exists. The schema files are generated directly off plugin.spec.yaml (source of truth) and should not be modified by the user. In this design, developers only need to modify the code files.

An example of a Python plugin that is V2 compliant:


 
1
base64
2
├── Dockerfile
3
├── Makefile
4
├── bin
5
│   └── komand_base64
6
├── komand-base64-0.2.2.tar.gz
7
├── komand_base64
8
│   ├── __init__.py
9
│   ├── actions
10
│   │   ├── __init__.py
11
│   │   ├── decode
12
│   │   │   ├── __init__.py
13
│   │   │   ├── action.py
14
│   │   │   └── schema.py
15
│   │   └── encode
16
│   │       ├── __init__.py
17
│   │       ├── action.py
18
│   │       └── schema.py
19
│   ├── connection
20
│   │   ├── __init__.py
21
│   │   ├── connection.py
22
│   │   └── schema.py
23
│   ├── triggers
24
│   │   └── __init__.py
25
│   └── util
26
│       └── __init__.py
27
├── plugin.spec.yaml
28
├── requirements.txt
29
├── run.sh -> ../tools/run.sh
30
├── setup.py
31
└── tests
32
    ├── decode.json
33
    └── encode.json

Plugins developed after October 1, 2016 have the ability to be regenerated using the older git method.
- V1 method of Regenerating Plugins
Plugins developed after January 1, 2018 or plugins that have been ported to V2 have the ability to be regenerated using the newer method.
- The plugin V2 architecture provides regeneration natively with icon-plugin generate --regenerate.
- All our plugins have been ported to V2 but your custom plugins may have not, if they haven't, it's best to port them first to make development easier for the long term.

Generation Command

As described in the SDK Spec document, plugin skeletons are generated by running:

icon-plugin generate python plugin.spec.yaml

Regeneration Command

If you decide to modify your plugin’s scheme, by editing plugin.spec.yaml, you’ll need to regenerate the plugin skeleton using make regenerate inside the plugin’s directory. The make regenerate target is a shortcut that calls the full regenerate command which is icon-plugin generate python --regenerate.

Process

The following process will automatically put the new schema changes from plugin.spec.yaml into effect:

Update the plugin.spec.yaml file with the new schema
Run make regenerate
Update any code files needed (e.g. action.py)

Example

Here's an example of plugin regeneration by updating the schema (plugin.spec.yaml) to support:

A new action to lookup malware hashes
A new plugin description
A new plugin version


 
1
$ vim plugin.spec.yaml # Updated the schema
2
$ make regenerate
3
icon-plugin generate python --regenerate
4
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/Makefile
5
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/.gitignore
6
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/setup.py
7
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/bin/komand_cymon
8
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/connection/schema.py
9
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/domain_blacklist/schema.py
10
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/domain_blacklist/__init__.py
11
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/malware_lookup/action.py
12
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/malware_lookup/schema.py
13
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/malware_lookup/__init__.py
14
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/domain_lookup/schema.py
15
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/domain_lookup/__init__.py
16
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/address_lookup/schema.py
17
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/address_lookup/__init__.py
18
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/url_lookup/schema.py
19
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/url_lookup/__init__.py
20
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/address_blacklist/schema.py
21
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/address_blacklist/__init__.py
22
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/actions/__init__.py
23
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/triggers/poll_address_blacklist/schema.py
24
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/triggers/poll_address_blacklist/__init__.py
25
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/triggers/poll_domain_blacklist/schema.py
26
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/triggers/poll_domain_blacklist/__init__.py
27
INFO[0000] Writing to file: /Users/jschipp/go/src/github.com/rapid7/insightconnect-plugins/cymon/komand_cymon/triggers/__init__.py
28
INFO[0000] Code regeneration complete!

Here's a list of the changes with git status. You can see that a few files were modified as well as the new addition of the malware_lookup action directory.


 
1
$ git status
2
On branch master
3
Your branch is up to date with 'origin/master'.
4

5
Changes not staged for commit:
6
  (use "git add <file>..." to update what will be committed)
7
  (use "git checkout -- <file>..." to discard changes in working directory)
8

9
      modified:   bin/komand_cymon
10
      modified:   komand_cymon/actions/__init__.py
11
      modified:   plugin.spec.yaml
12
      modified:   setup.py
13

14
Untracked files:
15
  (use "git add <file>..." to include in what will be committed)
16

17
      komand_cymon/actions/malware_lookup/

We can see the exact changes using git diff. Notice that the description and version were updated in multiple files, specifically: cymon/bin/komand_cymon and setup.py. The new action resulted in the creation of new directory komand_cymon/actions/malware_lookup/ which contains an action.py and schema.py file. The rest of the output are the references to the new action.


 
1
diff --git a/cymon/bin/komand_cymon b/cymon/bin/komand_cymon
2
index ccd4c109d..7aed3ab00 100755
3
--- a/cymon/bin/komand_cymon
4
+++ b/cymon/bin/komand_cymon
5
@@ -6,8 +6,8 @@ from komand_cymon import connection, actions, triggers
6

7
 Name = 'Cymon'
8
 Vendor = 'rapid7'
9
-Version = '0.1.4'
10
-Description = 'Cymon Open Threat Intelligence'
11
+Version = '1.0.0'
12
+Description = 'Cymon is the largest tracker of open-source security reports about phishing, malware, botnets and other malicious activities'
13

14

15
 class KomandCymon(komand.Plugin):
16
@@ -31,6 +31,8 @@ class KomandCymon(komand.Plugin):
17

18
         self.add_action(actions.DomainLookup())
19

20
+        self.add_action(actions.MalwareLookup())
21
+
22
         self.add_action(actions.UrlLookup())
23

24

25
diff --git a/cymon/komand_cymon/actions/__init__.py b/cymon/komand_cymon/actions/__init__.py
26
index 41ac95ec7..44a025ccb 100755
27
--- a/cymon/komand_cymon/actions/__init__.py
28
+++ b/cymon/komand_cymon/actions/__init__.py
29
@@ -3,4 +3,5 @@ from .address_blacklist.action import AddressBlacklist
30
 from .address_lookup.action import AddressLookup
31
 from .domain_blacklist.action import DomainBlacklist
32
 from .domain_lookup.action import DomainLookup
33
+from .malware_lookup.action import MalwareLookup
34
 from .url_lookup.action import UrlLookup
35
diff --git a/cymon/plugin.spec.yaml b/cymon/plugin.spec.yaml
36
index a435d9eaf..23dda41a8 100644
37
--- a/cymon/plugin.spec.yaml
38
+++ b/cymon/plugin.spec.yaml
39
@@ -1,8 +1,8 @@
40
 plugin_spec_version: v2
41
 name: cymon
42
 title: Cymon
43
-description: "Cymon Open Threat Intelligence"
44
-version: 0.1.4
45
+description: "Cymon is the largest tracker of open-source security reports about phishing, malware, botnets and other malicious activities"
46
+version: 1.0.0
47
 vendor: rapid7
48
 @@ -529,3 +529,14 @@ actions:
49
       url:
50
         type: "[]string"
51
         title: Cymon URL References
52
+  malware_lookup:
53
+    description: Lookup malware hash
54
+    input:
55
+      hash:
56
+        type: string
57
+        description: MD5 or SHA1 hash
58
+        required: true
59
+    output:
60
+      found:
61
+        type: boolean
62
+        title: Malware Found
63
diff --git a/cymon/setup.py b/cymon/setup.py
64
index e72761182..6e8117680 100755
65
--- a/cymon/setup.py
66
+++ b/cymon/setup.py
67
@@ -3,8 +3,8 @@ from setuptools import setup, find_packages
68

69

70
 setup(name='cymon-komand-plugin',
71
-      version='0.1.4',
72
-      description='Cymon Open Threat Intelligence',
73
+      version='1.0.0',
74
+      description='Cymon is the largest tracker of open-source security reports about phishing, malware, botnets and other malicious activities',
75
       author='komand',
76
       author_email='',
77
       url='',

In addition, you will see changes to the plugin.spec.yaml file itself, since it’s the source of truth and we created those changes manually.

Now, add your code to the new komand_cymon/actions/malware_lookup/action.py file, update the help section of the plugin spec (make help), test, and finalize your changes with:


 
1
$ git add .
2
$ git commit -m "Regenerate Cymon with new description and new malware lookup action"

Did this page help you?