Web traffic logs and data sources for Office 365 Cloud App Security

Office 365 Advanced Security Management is now Office 365 Cloud App Security.

Evaluation    >

Planning    >

Deployment    >

Utilization   

Start evaluating

Start planning

Start deploying

You are here!

Next steps

Note: Office 365 Cloud App Security is available in Office 365 Enterprise E5 or as an add-on for another Office 365 Enterprise subscription. To view or add to your subscription, as a global admin, sign in to Office 365, and then choose Admin > Billing. For more information about plan options, see Compare All Office 365 for Business Plans.

You can use a wide range of web traffic log files and data sources with Office 365 Cloud App Security. However, your web traffic log files must include specific information and be formatted a certain way so that they will work with Office 365 Cloud App Security app discovery reports and the Cloud Discovery dashboard. Use this article as a reference guide for the web traffic logs and data sources you'll use with Office 365 Cloud App Security.

Web traffic log requirements

Office 365 Cloud App Security uses data in your web traffic logs to help you understand which apps people in your organization are using. The more details that are included in the log files, the better visibility you'll have into user activity.

The following table lists the requirements and attributes that are needed for your web traffic logs to work correctly with Office 365 Cloud App Security:

Attributes

Additional requirements

  • Date of the transaction

  • Source IP

  • Source user (recommended)

  • Destination IP address

  • Destination URL (recommended: URLs provide higher accuracy for cloud app detection than IP addresses)

  • Total amount of data (recommended)

  • Amount of uploaded or downloaded data (recommended: provides insights about cloud app usage patterns)

  • Action taken (allowed or blocked)

  • The data source for the log files must be supported.

  • The format the log files use must match the standard format. When the file is uploaded, app discovery will verify this.

  • The events in the log must have taken place no more than 90 days ago.

  • The log file must include outbound traffic information that can be analyzed for network activity.

If attributes aren't included in the logs that are loaded, Office 365 Cloud App Security can't show or analyze the information for you. For example, Cisco ASA Firewall's standard log format does not include the amount of uploaded bytes per transaction, the username, or a target URL (only a target IP). Because that information isn't in the Cisco log files, Office 365 Cloud App Security won't include it when analyzing your organization's network traffic.

Note: For some kinds of firewalls, you must set an information level for web traffic logs to include the required attributes. For example, Cisco ASA firewalls must have the information level set to 6. Make sure to confirm that your firewalls are set to deliver the correct information in your web traffic logs.

Data attributes for different vendors

The following table summarizes the information in web traffic logs from various vendors. Be sure to check with your vendor for the most current information.

Data source

Target app URL

Target app IP

Username

Origin IP

Total traffic

Uploaded bytes

Barracuda

Yes

Yes

Yes

Yes

No

No

Blue Coat

Yes

No

Yes

Yes

Yes

Yes

Checkpoint

No

Yes

No

Yes

No

No

Cisco ASA

No

Yes

No

Yes

Yes

No

Cisco FWSM

No

Yes

No

Yes

Yes

No

Cisco Ironport WSA

Yes

Yes

Yes

Yes

Yes

Yes

Cisco Meraki

Yes

Yes

No

Yes

No

No

Clavister NGFW (Syslog)

Yes

Yes

Yes

Yes

Yes

Yes

Dell SonicWall

Yes

Yes

No

Yes

Yes

Yes

Fortigate

No

Yes

No

Yes

Yes

Yes

Juniper SRX

No

Yes

No

Yes

Yes

Yes

Juniper SSG

No

Yes

No

Yes

Yes

Yes

McAfee SWG

Yes

No

No

Yes

Yes

Yes

Meraki (Cisco)

Yes

Yes

No

Yes

No

No

Microsoft Threat Management Gateway

Yes

No

Yes

Yes

Yes

Yes

Palo Alto Networks

Yes

Yes

Yes

Yes

Yes

Yes

Sophos

Yes

Yes

Yes

Yes

Yes

No

Squid (Common)

Yes

No

Yes

Yes

No

Yes

Squid (Native)

Yes

No

Yes

Yes

No

Yes

Websense - Investigative detail report (CSV)

Yes

Yes

Yes

Yes

Yes

Yes

Websense - Internet activity log (CEF)

Yes

Yes

Yes

Yes

Yes

Yes

Zscaler

Yes

Yes

Yes

Yes

Yes

Yes

Supported vendor firewalls and proxies

Office 365 Cloud App Security supports the following firewalls and proxies.

  • Barracuda - Web App Firewall (W3C)

  • Blue Coat Proxy SG - Access log (W3C)

  • Check Point

  • Cisco ASA Firewall (note that you must set the information level to 6)

  • Cisco IronPort WSA

  • Cisco ScanSafe

  • Cisco Merkai – URLs log

  • Dell Sonicwall

  • Fortinet Fortigate

  • Juniper SRX

  • Juniper SSG

  • McAfee Secure Web Gateway

  • Microsoft Forefront Threat Management Gateway (W3C)

  • Palo Alto series Firewall

  • Sophos SG

  • Sophos Cyberoam

  • Squid (Common)

  • Squid (Native)

  • Websense - Web Security Solutions - Investigative detail report (CSV)

  • Websense - Web Security Solutions - Internet activity log (CEF)

  • Zscaler

Note: If a data source that you'd like to use is not included here, you can request that it be added to app discovery. To do that, when you're creating a report, select Other for Data source. Then type the name of the data source that you're trying to upload. We'll review the log, and let you know if we add support for that log type.

Troubleshoot errors when log files are uploaded

After you upload web traffic log files, check the governance log to see if there were any errors. If there are errors, use the information in the following table to resolve those errors.

Error

Description

Resolution

Unsupported file type

The file uploaded is not a valid log file. For example, an image file.

Upload a text, zip, or gzip file that was directly exported from your firewall or proxy.

Internal error

An internal resource failure was detected.

Click Retry to re-run the task.

The log format does not match

The log format you uploaded does not match the expected log format for this data source.

  1. Verify that the log is not corrupt.

  2. Compare and match the log file format to the sample format shown on the upload page.

Transactions are more than 90 days old

All transaction are more than 90 days old and therefore are being ignored.

Export a new log with recent events and re-upload it.

No transactions to catalogue cloud apps

No transaction to any recognized cloud apps are found in the log.

Verify that the log contains outbound traffic information.

Unsupported log type

When you select Data source = Other (unsupported), the log is not parsed. Instead, it is sent for review to the Microsoft Cloud App Security technical team.

The Microsoft Cloud App Security technical team builds a dedicated parser for each data source. Most popular data sources are already supported. When an unsupported data source is uploaded, it is reviewed and added to the list of potential new data source parsers.

When a new parser is added to the feature, a notification is included in the Microsoft Cloud App Security release notes.

Next steps

Connect with an expert
Contact us
Expand your skills
Explore training

Was this information helpful?

Thank you for your feedback!

Thank you for your feedback! It sounds like it might be helpful to connect you to one of our Office support agents.

×