Zscaler NSS

Zscaler is a software as a service (SaaS) web proxy with an "on-premises" NSS component that retrieves the logs from the cloud and pulls them into the local network for log aggregators, such as the InsightIDR Collector.

Zscaler NSS product logs can contain information about hosts and accounts, in addition to the source address. When setting up Zscaler NSS as an event source, you will have the ability to specify attribution options.

To set up Zscaler NSS, you’ll need to:

  1. Review “Before you Begin” and note any requirements,
  2. Configure Zscaler NSS to send data to your Collector,
  3. Set up the Zscaler NSS event source in InsightIDR, and
  4. Verify the configuration works.

Before You Begin

You must prepare Zscaler NSS to forward logs to the InsightIDR Collector.

You can find additional information on how to configure Zscaler NSS here: https://help.zscaler.com/zia/documentation-knowledgebase/analytics/nss/nss-deployment-guides.

InsightIDR supports Web, Firewall and DNS logs

While Zscaler NSS supports multiple log types, InsightIDR currently only has parsers for certain formats, depending on the type of log:

You must set up a separate Zscaler NSS event source for each NSS feed you’d like to send to InsightIDR. If you’d like to send Web, Firewall and DNS logs to InsightIDR, you should set up three Zscaler NSS event sources.

Configure Zscaler NSS to send data to your Collector

The Zscaler logs must arrive in a certain format depending on the type of log for InsightIDR to correctly parse them.

Forward Web Logs

In Zscaler, select LEEF as the format when setting up your NSS feed. Then, paste the following format into the Feed Output Format field:

%s{mon} %02d{dd} %02d{hh}:%02d{mm}:%02d{ss} zscaler-nss: LEEF:1.0|Zscaler|NSS|4.1|%s{reason}|cat=%s{action}\tdevTime=%s{mon} %02d{dd} %d{yy} %02d{hh}:%02d{mm}:%02d{ss} America/Chicago\tdevTimeFormat=MMM dd yyyy HH:mm:ss z\tsrc=%s{cip}\tdst=%s{sip}\tsrcPostNAT=%s{cintip}\trealm=%s{location}\tusrName=%s{login}\tsrcBytes=%d{reqsize}\tdstBytes=%d{respsize}\trole=%s{dept}\tpolicy=%s{reason}\turl=%s{url}\trecordid=%d{recordid}\tbwthrottle=%s{bwthrottle}\tuseragent=%s{ua}\treferer=%s{referer}\thostname=%s{host}\tappproto=%s{proto}\turlcategory=%s{urlcat}\turlsupercategory=%s{urlsupercat}\turlclass=%s{urlclass}\tappclass=%s{appclass}\tappname=%s{appname}\tmalwaretype=%s{malwarecat}\tmalwareclass=%s{malwareclass}\tthreatname=%s{threatname}\triskscore=%d{riskscore}\tdlpdict=%s{dlpdict}\tdlpeng=%s{dlpeng}\tfileclass=%s{fileclass}\tfiletype=%s{filetype}\treqmethod=%s{reqmethod}\trespcode=%s{respcode}

Forward Firewall Logs

In Zscaler, select JSON as the format when setting up your NSS feed. Then, paste the following format into the Feed Output Format field:

\{ "sourcetype": "zscalernss-fw", "event":\{"datetime":"%s{time}","user":"%s{login}","locationname":"%s{location}","cdport":"%d{cdport}","csport":"%d{csport}","csip":"%s{csip}","cdip":"%s{cdip}","action":"%s{action}","proto":"%s{ipproto}","inbytes":"%ld{inbytes}","outbytes":"%ld{outbytes}","deviceowner":"%s{deviceowner}","devicehostname":"%s{devicehostname}"\}\}

These are the minimum required fields needed to parse Firewall logs in InsightIDR. You can also add additional fields within the event section if you’d like to send other information.

Forward DNS Logs

In Zscaler, select JSON as the format when setting up your NSS feed. Then, paste the following format into the Feed Output Format field:

\{ "sourcetype": "zscalernss-dns", "event":\{"datetime":"%s{time}","user":"%s{login}","location":"%s{location}","reqaction":"%s{reqaction}","dns_reqtype":"%s{reqtype}","dns_req":"%s{req}","srv_dport":"%d{sport}","clt_sip":"%s{cip}","srv_dip":"%s{sip}"\}\}

These are the minimum required fields needed to parse DNS logs in InsightIDR. You can also add additional fields within the event section if you’d like to send other information.

Set up Zscaler NSS in InsightIDR

You can configure a Zscaler NSS event source for each NSS feed you’d like to send to InsightIDR.

  1. From your dashboard, select Data Collection on the left hand menu.
  2. When the Data Collection page appears, click the Setup Event Source dropdown and choose Add Event Source.
  3. From the “Security Data” section, click the Web Proxy icon. The “Add Event Source” panel appears.
  4. Choose your collector and event source. You can also name your event source if you want.
  5. Optionally choose to send unfiltered logs.
  6. Choose the timezone that matches the location of your event source logs.
  7. Select an attribution source.
  8. Select a collection method and specify a port and a protocol.
    • Optionally choose to Encrypt the event source if choosing TCP by downloading the Rapid7 Certificate.
  9. Click Save.

Attribution source options

ZScaler NSS product logs can contain information about hosts and accounts. When setting up ZScaler NSS as an event source, you will have the ability to specify the following attribution options:

  1. Use IDR engine if possible; if not, use event log

By selecting this option, the InsightIDR attribution engine will perform attribution using the source address present in the log lines. If it's unable to resolve assets or accounts using the source address, it will use the assets or accounts present in the log lines, if any.

  1. Use event log if possible; if not, use IDR engine

By selecting this option, attribution will be done using the assets and accounts present in the log lines. If no assets or accounts are present in the log lines, the InsightIDR attribution engine will perform attribution using the source address present in the log lines.

  1. Use IDR engine only

By selecting this option, the InsightIDR attribution engine will perform the attribution using the source address present in the log lines, ignoring any assets and accounts present in the log lines.

  1. Use event log only

By selecting this option, attribution will be done using the assets and accounts present in the log lines, ignoring the source address.

Verify the configuration

Complete the following steps to view your logs and ensure events are making it to the Collector:

  1. Click Data Collection in the left menu of InsightIDR and navigate to the Event Sources tab. Find the new event source that was just created and click the View Raw Log button. If you see log messages in the box, then this shows that logs are flowing to the Collector.
  2. Click Log Search in the left menu of InsightIDR.
  3. Select the applicable Log Sets and the Log Names within them. The Log Name will be the name you gave to your event source. Zscaler logs flow into the Web Proxy Activity log set when the log is generated from a web proxy event.

Please note that logs take at least 7 minutes to appear in Log Search after you set up the event source. If you see log messages when you select View Raw Log on the event source but do not see any log messages in Log Search after waiting for a few minutes for them to appear, then your logs do not match the recommended format and type for this event source.