Harvesting Strategies

InsightCloudSec comes with many default harvesting strategies so your accounts are harvested automatically at a cadence we recommend without any additional configuration. If you want to add, manage, or explore the harvesting strategies available in your InsightCloudSec instance, you can do so on the Harvesting Strategy page.

Hello, new experience!

With the version 24.10.1 release of InsightCloudSec, we've added a new version of the Harvesting Strategy page that improves navigation, filtering, configuration, and management experiences. The old interface (Legacy UI) is on by default, but you can click the Switch to Modern UI button to test out the new interface.

Legacy UI documentation

Explore harvesting strategies

You can view all existing harvesting strategies on the Harvesting Strategy page. Only users with appropriate permissions can access Harvesting Strategies from the Settings menu (gear icon).

To open the Harvesting Strategy page:

  1. Log in to InsightCloudSec.
  2. Click Settings > Harvesting Strategies (gear icon).

The following actions are available on the Harvesting Strategy page:

  • You can use the Cloud Support icons to narrow the results or you can use the Search bar to find a particular harvesting strategy.
  • You can delete a harvesting strategy at any time by clicking Delete strategy (trashcan icon).
  • You can make a strategy the default for a given cloud provider by clicking Make Default (checkmark icon).

Creating a Harvesting Strategy

Refer to the following steps to create a new harvesting strategy:

  1. Go to Administration > Harvesting Strategies and select New Strategy.
  2. Provide a name for your strategy, select the appropriate cloud service provider (e.g. AWS) and click Create.
  3. On the Strategy Configuration page, review and update the parameters as follows:
    • Region where a new strategy applies (from the Region drop-down box at the top right of the page).
    • Overall percentage change (faster or slower) to the default cadence (from the "Cadence" slide bar).
    • Overrides (in minutes) for specific resource types or harvesters. Default overrides (in minutes) are shown for reference.
    • Apply Normal will reapply the normal Harvest Strategy settings.
    • Apply Daily will apply an an override of 1440 minutes (24 hrs) to all of the settings.

    Override Settings

    Overrides cannot be less than 1 minute or greater than 24 hours.

    For AWS EDH-enabled resources and regions, InsightCloudSec sets a maximum harvesting frequency of four hours.

  4. Click Save & Copy to Regions to apply your strategy to other regions for easier configuration. (This applies when a region other than Global Harvest was initially selected.)

    Limiting Regions

    If you want to create a custom strategy or modify the default strategy in order to exclude a specific region or regions (e.g., working backwards by removing rather than adding regions), you can also create or copy an existing strategy and modify that strategy using the Admin option.

  5. After you have completed your changes to the entire strategy, click Save Changes.
  6. Click Harvesting Strategies to return to the main page and assign resources.
  7. Select the Assign (clipboard) icon (under the Admin options).
  8. In the Strategy Assignment window, select the cloud accounts to be harvested by this strategy. You can scope the strategy by cloud account or by badge.

    Any existing strategy previously assigned to selected accounts will be overwritten.

Modifying an Existing Strategy

Refer to the steps below to modify an existing strategy.

Permissions

Domain Admin permissions are required for the majority of these changes. Contact your administrator or reach out to us through the Customer Support Portal if you have issues or questions.

  1. Locate the strategy you want to modify from Administration > Harvesting Strategies on the Harvesting Strategy - Listing page. You can scope this list by cloud or by strategy name.
  2. From the Harvesting Strategy page under Admin, you can do the following:
    • Configure the strategy (wrench icon) - This navigates to the Strategy Configuration page
    • Assign (or reassign) - This strategy connects to a cloud account (note pad icon)
    • Make Default (check mark) - Will make the target strategy to default for the clouds selected as they are added; this option is available for each cloud type
    • Edit Strategy (pencil icon) - Allows you to modify the strategy name and exclude specific regions from harvesting. Currently, the ability to limit regions and mark resources for deletion in disabled regions is only available for AWS.
    • Delete the strategy - Allows you to delete the strategy with the appropriate permissions and confirmation.
  3. If you opt to configure an existing strategy, your options are the same as those available for creating a New Strategy.
    • Region where a new strategy applies (from the Region drop-down box at the top right of the page).
    • Overall percentage change (faster or slower) to the default cadence (from the Cadence slide bar).
    • Overrides (in minutes) for specific resource types or harvesters. Default overrides (in minutes) are shown for reference.
    • Apply Normal will reapply the normal Harvest Strategy settings.
    • Apply Daily will apply an an override of 1440 minutes (24 hrs) to all of the settings.
  4. Make any changes desired and save.

Selecting Harvesting Strategy for Newly Added Accounts

Once a strategy has been set up, an administrator can assign any cloud account to it, including new accounts as they are added. For details on adding a new cloud account, refer to details on the Cloud Account Setup page. If you don't specify a harvesting strategy when adding the cloud account, it will be assigned to the default strategy for that cloud provider.

  1. When adding a cloud account via the UI, click Show Advanced (at the bottom) and a drop-down menu will display strategies applicable to the type of cloud you are adding.
  2. Click the desired strategy and select Submit when the remainder of the Add Cloud pane is complete. The Harvesting Strategy drop-down option will only appear for organizations that have additional harvesting strategies configured; otherwise, the default is applied and no drop-down menu appears.

Harvesting Strategy Example

In the following example, harvesting times are slowed for Amazon Web Services (AWS) outside the Continental US.

  1. First, we create a new harvesting strategy named New Strategy for AWS.
  2. Second, we decrease the cadence of all harvesting in one region by the following:
    • Choosing one region outside the US (ap-northeast-1).
    • Decreasing the overall harvesting cadence; in this case, we have decreased the cadence by 1000% from default values.
    • We can also override harvesting times for specific resources by entering those numbers (minutes) into the boxes next to the specific resource.
  3. Select Apply to save the changes.
  4. Next, we apply this newly created strategy to all regions by copying that strategy to everything except regions within the Continental US (us-east-1 and 2, us-west-1 and 2).
    • We've used the copy icon, next to the Region box, to open the Region to Copy to pane and select (or deselect) the appropriate regions.
  5. Finally, we return to the Harvesting Strategy listing to assign cloud accounts to the new strategy.
    • On the first return to this listing, you will see that New Strategy is assigned to no clouds.
    • Clicking the clipboard icon opens the Assign Strategy pane, where we have added one of our AWS cloud accounts.
Modern UI documentation

Explore harvesting strategies

You can view all existing harvesting strategies on the Harvesting Strategy page. Only users with appropriate permissions can access Harvesting Strategies from the Settings menu (gear icon).

To open the Harvesting Strategy page:

  1. Log in to InsightCloudSec.
  2. Click Settings > Harvesting Strategies (gear icon).

The following actions are available on the Harvesting Strategy page:

  • You can use the Cloud Provider filter to narrow the results or you can use the Search bar to find a particular harvesting strategy.
  • You can delete a harvesting strategy at any time by clicking Action (ellipsis) > Delete Strategy > Delete.
  • You can make a strategy the default for a given cloud provider by clicking Action (ellipsis) > Make Default > Make Default.

Create a harvesting strategy

If you would like to create a custom harvesting strategy for a particular cloud account or group of accounts, you can create a new one from the Harvesting Strategy page.

To create a new harvesting strategy:

  1. From the Harvesting Strategies page, click + New strategy.
  2. Enter a Strategy Name.
  3. Select a Cloud Provider from the drop-down menu. Only one cloud provider can be selected.
  4. Click Create.

The harvesting strategy is created with dynamic scheduling turned on. Dynamic scheduling means that harvesting job (also known as harvesters) frequency is dynamically determined based on the resources the selected cloud account has. For example: if an account has no compute instances associated with it, InsightCloudSec automatically adjusts the compute instance harvesting job to occur once every 24 hours. Explore modifying a strategy to adjust its regions, scheduling, and settings.

Modify a harvesting strategy

Modifying a harvesting strategy can consist of the following activities:

Configure a harvesting strategy

You can configure a harvesting strategy's job frequency globally (all regions) or per region. You can also configure a harvesting job's frequency if dynamic scheduling is not turned on. To pause region harvesting or toggle dynamic scheduling, you must edit the strategy instead.

To configure individual harvesters for a region:

  1. From the Harvesting Strategies page, click Action (ellipsis) > Configure Strategy next to the harvesting strategy you want to configure.
  2. Select the Region you want to edit. Global is the default selection and will override individual region settings.
  3. For each harvester you want to update the frequency override (use the search bar to filter the list):
    1. Click Add Override.
    2. Enter a new frequency in minutes noting the default setting.
    3. Click Apply.
  4. Save your changes:
    1. Optionally, if you edited a particular region (not global), you can click Save & Copy to Regions to also apply the frequency overrides to additional regions.
    2. If you only want your changes to apply to one region, click Save Changes.

To modify the frequency for an entire region:

  1. From the Harvesting Strategies page, click Action (ellipsis) > Configure Strategy next to the harvesting strategy you want to configure.
  2. Select the Region you want to edit. Global is the default selection and will override individual region settings.
  3. Select an option in the Modify Frequency drop-down menu:
    • Default Frequency - Sets all harvester frequencies to the default value for the region.
    • Daily Frequency - Sets all harvester frequencies to 1440 minutes (24 hours) for the region.
    • Custom - Sets all harvester frequencies to a value relative to the percentage selected from a slider (-2500% slower to +500% faster).
  4. Click Apply. If you're not happy with the changes, you can click Restore Defaults.
  5. Save your changes:
    1. Optionally, if you edited a particular region (not global), you can click Save & Copy to Regions to also apply the frequency overrides to additional regions.
    2. If you only want your changes to apply to one region, click Save Changes.
Assign a strategy

You can assign a strategy to cloud accounts from the Harvesting Strategies page instead of individually updating an account's settings.

To assign a strategy:

  1. From the Harvesting Strategies page, click Action (ellipsis) > Assign Strategy next to the harvesting strategy you want to configure.
  2. Use the Select Cloud Accounts drop-down to select cloud accounts to assign the strategy to.
  3. Optionally, select Assign Strategy to clouds with badges to show a Select Badges drop-down menu where you can select cloud badges to apply the strategy to all cloud accounts with a selected badge.
  4. Click Assign. If a cloud account already had a strategy, it will be overridden.
Edit a strategy

You can edit a strategy to change its name, toggle dynamic scheduling, pause harvesting regions (AWS only), or mark resources in disabled regions for deletion (AWS only).

To edit a strategy:

  1. From the Harvesting Strategies page, click Action (ellipsis) > Edit Strategy next to the harvesting strategy you want to configure.
  2. Toggle dynamic scheduling using the Enable Dynamic Scheduling checkbox.
  3. For AWS harvesting strategies;
    1. Optionally, update the Select Regions drop-down menu with regions to pause harvesting for.
    2. Optionally, select the Mark disabled regions Resources for deletion checkbox.
  4. Click Save.