Microsoft ActiveSync and Outlook Web Access
Adding in email and ActiveSync logs helps Insight track your user’s devices, track user locations with ActiveSync and OWA, and investigate malicious links from emails.
Rapid7's monitoring of OWA/ActiveSync activity understands that these are IIS web applications. Therefore, you can configure a directory watcher on the collector to monitor the IIS logs of the computer running the Exchange software, and look for web requests that match OWA/ActiveSync signatures.
InsightIDR requires a public IP address in the log to generate an Ingress event and parse Ingress Activity.
Mobile logons via wireless networks are on ingress map
Mobile provider geoips do not show up on your ingress activity map because the geolocation for these IPs is usually inaccurate. Mobile logons via wireless networks will still show up on your ingress map.
To set up OWA/ActiveSync, you’ll need to:
- Review “Before you Begin” and note any requirements.
- Configure OWA/ActiveSync to send data to your Collector.
- Set up the OWA/ActiveSync event source in InsightIDR.
- Verify the configuration works.
Additionally:
Before You Begin
You'll need the following to use the OWA/ActiveSync event source:
- Ensure there are no devices proxying the connection between the external endpoint and ActiveSync and the ActiveSync server.
- If there is device proxying the connection between the external endpoint and the ActiveSync server, you need a x-forwarded-for header. This provides the source IP for the external endpoint by the device that is doing the proxying. Since a proxy is in between the two devices, and it is the proxy connecting to the ActiveSync server directly rather than the external endpoint, the s-ip field will have the proxy's IP.
A load balancers’s effect on IP addresses for OWA/ActiveSync
If you have a load balancer like Netscaler in front of your OWA/Exchange servers, you may experience that the source IP for all users is the load balancer instead of the true IP address. Review the troubleshooting steps below to prevent issues.
Configure OWA/ActiveSync to support data retrieval by your Collector
You must prepare your OWA and ActiveSync servers to support data collection by Collectors.
In order to have the Collector ingest logs from Microsoft Outlook Web Access (OWA) and ActiveSync services, perform the following steps on the server side:
- Determine the destination folder for the logs that the Internet Information Services (IIS) process responsible for running OWA/ActiveSync generates.
- Note: You cannot have logs nested in folders.
- Ensure that the IIS process logs the expected fields to the log files.
- Share the log folder with a read credential that is also to be entered in InsightIDR.
How to configure Internet Information Services (IIS)
Perform the following steps:
- Determine which server is responsible for handling the OWA/ActiveSync client requests you would like to be gathered by InsightIDR.
- On that server, launch the IIS Manager from the "Start" menu.
- Click the Logging icon in the IIS Manager.
- The Logging module displays where the IIS logs are recorded as well as how to specify the exact fields to log. Make a note of the log folder because you will need to enter this folder in the InsightIDR event source.
- Click the Select Fields button to select the appropriate fields to log.
The fields selected for the log file must exactly match those shown in the following list, including the order:
date
time
s-ip
cs-method
cs-uri-stem
cs-uri-query
s-port
cs-username
c-ip
cs(User-Agent)
sc-status
sc-substatus
sc-win32-status
x-forwarded-for
This is how they look in the log file:
1date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status x-forwarded-for
- Click the OK button to save your changes.
This is the log entry that gets written to the start of every log file upon log rotation:
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status X-Forwarded-For
Windows file system configuration
Follow these steps to configure the log folder to allow the Collector to reach the logs:
- In Windows Explorer, right-click on the IIS log folder and click Properties.
- In Properties under Advanced Sharing, click Share this folder, then click the Permissions button.
- Click Add and provide the credential that will have access to this directory. The user name and password for this credential will also be entered in InsightIDR when the OWA/ActiveSync event source is set up.
Set up OWA/ActiveSync in InsightIDR
You can configure the OWA event source to read the shared folder via UNC notation and by providing the credential that was used when setting up the shared folder. UNC notation is Microsoft's Universal Naming Convention which is a common syntax used to describe the location of a network resource.
To configure the new event source in InsightIDR:
- From the left menu, go to Data Collection and click Setup Event Source > Add Event Source.
- Do one of the following:
- Search for Microsoft ActiveSync & Outlook Web Access in the event sources search bar.
- In the Product Type filter, select Email & ActiveSync.
- Select the Microsoft ActiveSync & Outlook Web Access event source tile.
- Choose your collector. You can also name your event source if you want.
- Optionally choose to send unparsed logs.
- Configure your default domain and any Advanced Event Source Settings.
- Select Watch Directory as your collection method.
- Click Save.
Format Logs to ELFF Format
The different logging formats for IIS logs are detailed in this documentation: https://msdn.microsoft.com/en-us/library/ms525807(v=vs.90).aspx.
InsightIDR only supports W3C ELFF format for IIS logs. The logs must have the correct fields in the order specified above. Any additional fields can be in the logs, but must come after the sc-win32-status
field.
Additional fields will not work for IIS version 8.5.
Verify the configuration
Complete the following steps to view your logs and ensure events are making it to the Collector:
- Click Data Collection in the left menu of InsightIDR and navigate to the Event Sources tab. Find the new event source that was just created and click the View Raw Log button. If you see log messages in the box, then this shows that logs are flowing to the Collector.
- Click Log Search in the left menu of InsightIDR.
- Select the applicable Log Sets and the Log Names within them. The Log Name will be the name you gave to your event source.
Logs take a minimum of 7 minutes to appear in Log Search
Please note that logs take at least 7 minutes to appear in Log Search after you set up the event source. If you see log messages when you select View Raw Log on the event source but do not see any log messages in Log Search after waiting for a few minutes for them to appear, then your logs do not match the recommended format and type for this event source.
Troubleshoot parsing issues with OWA/ActiveSync
If the data from OWA or ActiveSync is not parsing, you can review the logs to identify the cause of the issue. The most common issues are:
- The fields are not correctly configured
- Load balancer is set as a source IP
- There is no public IP Address in the logs
Specific issues that depend on the version you are using:
Once you identify the issue, you can make the necessary changes in the Exchange servers to solve it.
How to review OWA or ActiveSync logs to identify a parsing issue
You can review raw logs or do a log search:
To review raw logs:
- Go to Data collection > Event Sources
- Select the View raw log option below the event source
To do a log search:
- Click the Log Search option in the left menu
- Search for the the OWA/ActiveSync event source in the Filter by Event source or type filter field
- If you can't see any logs for that event source, go to Data collection > Event Sources
- Select the Edit option for the event source you are reviewing and see if the unfiltered logs option is checked (You should repeat this steps and click the option again after solving the issue)
- Go back to Log Search to review the logs for this event source
- Use the search bar to search
where(fields,loose)
within the logs of the event source
Within the OWA or ActiveSync logs, you can follow the steps below to review them and identify the parsing issue. You can then make the necessary changes in the exchange servers to solve it. Review the common issues below to analyse which actions apply to you.
The fields are not correctly configured
A common reason why a parsing issue may occur is that the fields are not correctly configured. The fields selected for the log file must exactly match those shown in the following list, including the order:
date
time
s-ip
cs-method
cs-uri-stem
cs-uri-query
s-port
cs-username
c-ip
cs(User-Agent)
sc-status
sc-substatus
sc-win32-status
x-forwarded-for
This is how they should look like in the log file:
1date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status x-forwarded-for
This is the log entry that gets written to the start of every log file upon log rotation:
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status X-Forwarded-For
You can also compare the OWA or ActiveSync logs to sample logs.
Load balancer
If you have a load balancer, such as Netscaler, in front of your OWA/Exchange servers, you may experience that the source IP for all users is the load balancer instead of the true IP address.
To fix this, you must add an x-forwarded
header to the IIS logs. You can learn more about how to do this here: http://www.loadbalancer.org/blog/iis-and-x-forwarded-for-header/.
No Public IP Address
The IP addresses need to show at least one public IP for the s-ip
or x-forwarded-for
header. If you are having issues, review the configuration and check that no internal addresses are included instead of a public IP Address.
OWA/ActiveSync works by parsing Ingress Activity. To generate an Ingress event a public IP address should be in the log. If you have Exchange servers behind load balancers, then the true clientIP will need to be passed through to the Exchange server in order to get this to work correctly.
If there is an MDM, load balancer, or some other device between the external endpoint connecting to ActiveSync and the ActiveSync server, the "source IP" in the IIS logs for ActiveSync will be wrong. This is because it will point to the source IP of the intermediate device rather than the true source IP of the external endpoint. Therefore, you won't get any ingress activity on your map.
All these appliances have their own unique way of providing the true source IP in a custom HTTP request field. To fix this, complete the following:
- Go to the Exchange server to configure it for advanced logging and configure the advanced logs to match exactly the basic logs.
- Substitute the source IP (which will be the intermediate appliance, in this case) with the new field the appliance has added which represents the true source IP of the external endpoint.
Enhanced Logging for IIS 8.5+
Advanced Logging for IIS 7 is no longer available. Microsoft recommends using Enhanced Logging for IIS 8.5 instead.
Follow the instructions provided by Load Balancer to configure enhanced logging here: http://www.loadbalancer.org/blog/iis-and-x-forwarded-for-header/.
You can learn about enhanced logging here: https://www.iis.net/learn/get-started/whats-new-in-iis-85/enhanced-logging-for-iis85.
The field ordering is the following:
date
time
s-ip
cs-method
cs-uri-stem
cs-uri-query
s-port
cs-username
c-ip
cs(User-Agent)
sc-status
sc-substatus
sc-win32-status
x-forwarded-for
This is how they look like in the log file:
1date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status x-forwarded-for
InsightIDR now parses the additional field appended to the end of the log as the true Client IP address (c-ip).
Parsing examples
12013-04-03 18:50:42 11.2.33.44 POST /Microsoft-Server-ActiveSync/default.eas User=jsmith&DeviceId=ApplF2LJR67BDTTQ&DeviceType=iPhone&Cmd=Sync&Log=V141_Fc1_Fid:184_Ty:Em_Filt4_St:S_Sk:1563581951_Sst4_SsCmt4_Srv:1a0c0d0s0e0r0A0sd_BR1_BPR0_LdapC10_LdapL78_RpcC78_RpcL343_Pk4126865394_S1_As:AllowedG_Mbx:PORTLANDEX.tor.razor.com_Dc:CERVANTES.tor.razor.com_Throttle0_Budget:(A)Conn%3a0%2cHangingConn%3a0%2cAD%3a%24null%2f%24null%2f1%25%2cCAS%3a%24null%2f%24null%2f3%25%2cAB%3a%24null%2f%24null%2f0%25%2cRPC%3a%24null%2f%24null%2f0%25%2cFC%3a1000%2f0%2cPolicy%3aDefaultThrottlingPolicy%5Fe6b81fee-9b12-4f8f-adfb-930df7769654%2cNorm_ 443 tor\\jsmith 216.55.6.00 Apple-iPhone5C1/1002.143 200 0 0 16770232013-04-03 18:50:42 11.2.33.44 POST /Microsoft-Server-ActiveSync/Proxy/default.eas User=jsmith&DeviceId=ApplF2LJR67BDTTQ&DeviceType=iPhone&Cmd=Sync&Log=V141_Fc1_Fid:184_Ty:Em_Filt4_St:S_Sk:1563581951_Sst4_SsCmt4_Srv:1a0c0d0s0e0r0A0sd_BR1_BPR0_LdapC10_LdapL78_RpcC78_RpcL343_Pk4126865394_S1_As:AllowedG_Mbx:PORTLANDEX.tor.razor.com_Dc:CERVANTES.tor.razor.com_Throttle0_Budget:(A)Conn%3a0%2cHangingConn%3a0%2cAD%3a%24null%2f%24null%2f1%25%2cCAS%3a%24null%2f%24null%2f3%25%2cAB%3a%24null%2f%24null%2f0%25%2cRPC%3a%24null%2f%24null%2f0%25%2cFC%3a1000%2f0%2cPolicy%3aDefaultThrottlingPolicy%5Fe6b81fee-9b12-4f8f-adfb-930df7769654%2cNorm_ 443 tor\\jsmith 216.55.6.00 Apple-iPhone5C1/1002.143 200 0 0 16770452013-04-03 18:50:42 11.2.33.44 POST /Microsoft-Server-ActiveSync/default.eas Cmd=FolderSync&User=tor%5Cjsmith&DeviceId=androidc1474737936&DeviceType=Android&Log=PrxTo:laxex.tor.razor.com_LdapC7_LdapL30_Mbx:LAXEX.tor.razor.com_Dc:CERVANTES.tor.razor.com_Budget:(D)Conn%3a2%2cHangingConn%3a0%2cAD%3a%24null%2f%24null%2f1%25%2cCAS%3a%24null%2f%24null%2f0%25%2cAB%3a%24null%2f%24null%2f0%25%2cRPC%3a%24null%2f%24null%2f0%25%2cFC%3a1000%2f0%2cPolicy%3aDefaultThrottlingPolicy%5Fe6b81fee-9b12-4f8f-adfb-930df7769654%2cNorm%5bResources%3a(DC)CERVANTES.tor.razor.com(Health%3a-1%25%2cHistLoad%3a0)%2c%5d_ 443 tor\\jsmith 206.29.182.222 Android/4.2.2-EAS-1.3 200 0 0 12901672013-02-13 00:31:27 11.2.33.44 POST /owa/auth.owa - 443 tor\\jsmith 64.134.47.222 Mozilla/5.0+(Macintosh;+Intel+Mac+OS+X+10_8_2)+AppleWebKit/537.17+(KHTML892013-02-13 01:03:06 11.2.33.44 GET /owa/auth.owa - 443 tor\\jsmith 68.100.77.222 Mozilla/5.0+(compatible;+MSIE+9.0;+Windows+NT+6.1;+WOW64;+Trident/5.0) 401 1 0 12510112013-10-29 01:53:25 POST /Microsoft-Server-ActiveSync/default.eas Cmd=Sync&User=tor%5Caguerlain&DeviceId=SEC1325376102856&DeviceType=SAMSUNGSCHI535&Log=V141_Fc1_Fid:8_Ty:Em_Filt4_Filts4_St:S_Sk:1376563250_Sst40_SsCmt40_BR1_BPR0_LdapC1_RpcC44_RpcL297_Ers1_Pk535727841_S1_As:AllowedG_Mbx:RNEXCH2.ad.corp.local_Throttle0_Budget:(A)Conn%3a0%2cHangingConn%3a0%2cAD%3a%24null%2f%24null%2f0%25%2cCAS%3a%24null%2f%24null%2f1%25%2cAB%3a%24null%2f%24null%2f0%25%2cRPC%3a%24null%2f%24null%2f1%25%2cFC%3a1000%2f0%2cPolicy%3aDefaultThrottlingPolicy%5Ff0329dbe-9973-4fd7-8c2f-e966659fef23%2cNorm_ tor\\jsmith 192.168.44.44 SAMSUNG-SCH-I535/100.40102 - - 200 75 532 34312132013-10-29 02:18:08 POST /owa/auth.owa - tor\\jsmith 192.168.44.44 Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.36+(KHTML1415<13> mx2 2014-03-13 21:57:17 10.0.10.216 POST /Microsoft-Server-ActiveSync/default.eas Cmd=Sync&User=jbrown%5C&DeviceId=SEC11663A183923C&DeviceType=SAMSUNGGTI9300&Log=V141_Fc1_Fid:1_Ty:Ca_Filt0_St:S_Sk:1732773711_Sst247_SsCmt247_BR1_BPR0_LdapC1_RpcC51_RpcL31_Ers1_Pk2123537141_S1_As:AllowedG_Mbx:mx2.jbrown.com_Throttle0_Budget:(A)Conn%3a0%2cHangingConn%3a0%2cAD%3a%24null%2f%24null%2f0%25%2cCAS%3a%24null%2f%24null%2f1%25%2cAB%3a%24null%2f%24null%2f0%25%2cRPC%3a%24null%2f%24null%2f1%25%2cFC%3a1000%2f0%2cPolicy%3aDefaultThrottlingPolicy%5Fe4462822-4238-4dbb-bc78-d3e826f3e032%2cNorm_ 443 tor\\jsmith 199.47.66.66 SAMSUNG-GT-I9300/100.40101 200 0 0 20121617PHX-PWP-EXCAS01.razor.com\t\t0\t2014-03-17 15:39:04 10.99.33.222 POST /Microsoft-Server-ActiveSync/default.eas Cmd=Sync&User=jsmithsmashbros.com&DeviceId=SAMSUNG3A000002F298C94&DeviceType=SAMSUNGSPHD710&Log=V141_Fc1_Fid:RI_Ty:Ri_Filt0_St:S_Sk:1105471999_Sst10_Sslc7_BR0_BPR0_LdapC1_RpcC22_RpcL15_Ers1_Pk1325283713_S1_As:AllowedG_Mbx:PPP-EXCH1.razor.com_Throttle0_Budget:(A)Conn%3a0%2cHangingConn%3a0%2cAD%3a%24null%2f%24null%2f0%25%2cCAS%3a%24null%2f%24null%2f1%25%2cAB%3a%24null%2f%24null%2f0%25%2cRPC%3a%24null%2f%24null%2f1%25%2cFC%3a1000%2f0%2cPolicy%3aDefaultThrottlingPolicy%5F6e4556fa-9c9a-4706-8858-0994bfead688%2cNorm_ 443 jsmith@smashbros.com 66.87.99.111 SAMSUNG-SPH-D710/100.40004 200 0 1236 187511819PHX-PWP-EXCAS01.razor.com\tIISWebLog\t0\t2014-03-17 15:39:04 10.99.33.222 POST /Microsoft-Server-ActiveSync/default.eas Cmd=Sync&User=jsmithsmashbros.com&DeviceId=SAMSUNG3A000002F298C94&DeviceType=SAMSUNGSPHD710&Log=V141_Fc1_Fid:RI_Ty:Ri_Filt0_St:S_Sk:1105471999_Sst10_Sslc7_BR0_BPR0_LdapC1_RpcC22_RpcL15_Ers1_Pk1325283713_S1_As:AllowedG_Mbx:PPP-EXCH1.razor.com_Throttle0_Budget:(A)Conn%3a0%2cHangingConn%3a0%2cAD%3a%24null%2f%24null%2f0%25%2cCAS%3a%24null%2f%24null%2f1%25%2cAB%3a%24null%2f%24null%2f0%25%2cRPC%3a%24null%2f%24null%2f1%25%2cFC%3a1000%2f0%2cPolicy%3aDefaultThrottlingPolicy%5F6e4556fa-9c9a-4706-8858-0994bfead688%2cNorm_ 443 jsmith@smashbros.com 66.87.99.111 SAMSUNG-SPH-D710/100.40004 200 0 1236 1875120212014-07-14 19:09:58 172.16.111.1 POST /owa/auth.owa - 443 jsmith 14.0.77.55 Mozilla/5.0+(compatible;+Konqueror/3.5;+Linux)+KHTML/3.5.3+(like+Gecko) https://10.1.1.222/owa/auth/logon.aspx 302 0 1 622232014-07-14 19:08:18 fe80::d07b:d1c2:d57e:b06e%12 POST /owa/auth.owa - 443 mailbox@test.info ::1 Mozilla/4.0+(compatible;+MSIE+9.0;+Windows+NT+6.1;+MSEXCHMON;+ACTIVEMONITORING) - 302 0 1 224252014-07-29 15:49:36.817 10.10.133.111 POST /ews/exchange.asmx - 80 \26272014-07-29 16:09:13.740 10.10.133.111 POST /Microsoft-Server-ActiveSync/default.eas Cmd=Sync&DeviceId=9867045AB7356F2D99A7A3D8FAC9AEF3&DeviceType=WP8 80 \28292015-01-26 19:00:30.753 10.200.3.111 POST /Microsoft-Server-ActiveSync/default.eas User=xfoo&DeviceId=ApplC39M35K9FNJN&DeviceType=iPhone&Cmd=Sync 443 - \30312014-07-14 19:09:58 172.16.111.2 POST /owa/auth.owa - 443 jsmith 14.0.77.55 Mozilla/5.0+(compatible;+Konqueror/3.5;+Linux)+KHTML/3.5.3+(like+Gecko) https://10.1.1.222/owa/auth/logon.aspx 302 0 1 6232332016-03-07 17:27:55 10.10.111.55 GET /owa/ &CorrelationID=<empty>;&ClientId=BKHYY9ZIO0YI0UBKDDQ&cafeReqId=40407632-c626-4a4b-9131-f52b6e90b2e6; 443 TOR\\jsmith 75.98.77.111 Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.36+(KHTML34352016-04-22 02:18:54.129 169.254.1.111 POST /Microsoft-Server-ActiveSync/Proxy/default.eas User=jsmith&DeviceId=ApplDMPLMDKUFK12&DeviceType=iPad&Cmd=Sync 444 - \36372018-05-30 14:19:56 172.17.222.22 POST /owa/service.svc action=UpdateUserConfiguration 443 ajones 172.17.222.222 Mozilla/5.0+(Windows+NT+6.1)+AppleWebKit/537.36+(KHTML38392018-12-08 16:44:53 POST /Microsoft-Server-ActiveSync/default.eas User=parks.rec&DeviceId=ApplC39M35K9FNJN&DeviceType=iPhone&Cmd=Ping&CorrelationID=<empty>;&cafeReqId=40407632-c626-4a4b-9131-f52b6e90b2e6; 443 ssl\\parks.rec 185.150.222.222 HTTP/2.0 Apple-iPhone10C4/1505.302 - mail.ssl.ca 200 0 0 10221040412018-12-08 16:44:55 POST /Microsoft-Server-ActiveSync/default.eas User=jjs.diner&DeviceId=A35312OUHESDOFH0&DeviceType=iPhone&Cmd=Ping&CorrelationID=<empty>;&cafeReqId=40407632-c626-4a4b-9131-f52b6e90b2e6; 443 ssl\\jjs.diner 10.110.1.22 HTTP/2.0 Apple-iPhone10C4/1601.404 - mail.ssl.ca 200 0 0 3342042432019-08-29 23:59:54 10.1.22.22 POST /Microsoft-Server-ActiveSync/default.eas Cmd=Ping&User=a.user%40razor.com&DeviceId=PIEDPIPER54321&DeviceType=HOOLI1234&Log=V141_LdapC1_RpcC45_RpcL16_Hb470_S3_Error:PingCollisionDetected_Mbx:NLDC01VS086.RAZOR.LOCAL_Throttle0_Budget:(A)Conn%3a1%2cHangingConn%3a0%2cAD%3a%24null%2f%24null%2f0%25%2cCAS%3a%24null%2f%24null%2f1%25%2cAB%3a%24null%2f%24null%2f0%25%2cRPC%3a%24null%2f%24null%2f1%25%2cFC%3a1000%2f0%2cPolicy%3aDefaultThrottlingPolicy%5F3558ccbc-d46c-4173-bcc5-1342db72ac72%2cNorm_ 443 a.user@razor.com 11.11.11.11 - - 200 0 0 77.77.77.7744452019-08-30 00:01:43 10.1.22.22 GET /owa/auth/logon.aspx url=https://webmail.razor.com/owa/&reason=0 443 - 11.11.11.11 - - 200 0 0 77.77.77.7746472019-08-30 00:01:43 10.1.22.22 GET /owa/auth/logon.aspx url=https://webmail.razor.com/owa/&reason=0 443 - 11.11.11.11 - - 200 0 0 -48492020-06-24 11:19:01 192.82.111.6 POST /owa/ev.owa2 ns=PendingRequest&ev=FinishNotificationRequest&UA=0&ClientId=RMDMXVDECJYTWFPPCMKG&ActID=2a11a1d8-6397-4c54-a536-67472264e86d&CorrelationID=<empty> 444 tor\jsmith 192.82.111.6 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML