Create app discovery reports in Advanced Security Management

Office 365 productivity app discovery helps you understand which other cloud services your users are connecting to and using. For example, you can see where users are storing and collaborating on documents and how much data is being uploaded to apps or services outside of Office 365.

To generate a report, you must manually upload log files from your firewalls and proxies, and then app discovery will parse and analyze the files for the report.

What do you want to do?

Create a report with app discovery

Learn about log format requirements and supported data sources

Troubleshoot errors when log files are uploaded

Create a report with app discovery

To create an app discovery report, you identify the vendor data source for the log files that you want to have analyzed, select the log files, and then request the report. App discovery uploads and parses the files, analyzes them, and generates the report.

Follow these steps to create an app discovery report.

  1. Collect log files from the firewall and proxy that people in your organization use to access the Internet. Try to use log files from peak traffic periods that well represent the user activity in your organization.

  2. Sign in to Office 365, and go to the Security & Compliance Center.

  3. Choose Search & Investigation > Cloud app discovery.

  4. From the Discover menu, select Create new report.

  5. Type a name and description for the report you want to create, then select the data source (the firewall or proxy that generated your log files) from the drop-down.

    Tip: If a data source that you'd like to use is not listed, you can request that it be added. Select Other for Data source, and then type the name of the data source that you're trying to upload. We'll review the log, and let you know if we add support for the data source that generated it.

  6. Browse to the location of the log files you collected and select the files. The log files must have been generated by the data source that you chose for the report.

  7. Click Create to start the report creation process.

  8. To see the status of the report, click Manage snapshot reports. When the report is ready, you'll see the View option.

If there were any problems uploading or processing the log files, review the troubleshooting information for tips on how to figure out what went wrong and correct any issues.

Learn about log format requirements and supported data sources

App discovery supports a number of firewalls and proxies as data sources for reports (see the complete list below).

The web traffic log files from these sources must include specific information and be formatted in the right way so that app discovery can interpret the files and create reports.

Required format for traffic logs

App discovery looks at the data in your traffic logs to help you understand what apps your users are connecting to. The more details that are included in the log files, the better visibility you'll have into user activity.

App discovery requires traffic log files that includes the following:

  • Date of the transaction

  • Source IP

  • Source user (recommended)

  • Destination IP address

  • Destination URL (recommended: URLs provide higher accuracy for cloud app detection than IP addresses)

  • Total amount of data (recommended)

  • Amount of uploaded or downloaded data (recommended: provides insights about cloud app usage patterns)

  • Action taken (allowed or blocked)

If attributes aren't included in the logs that are loaded, app discovery can't show or analyze the information for you. For example, Cisco ASA Firewall's standard log format does not include the amount of uploaded bytes per transaction, the username, or a target URL (only a target IP). Because that information isn't in the Cisco log files, app discovery can't use it when analyzing the network traffic, so what you can learn about the cloud apps usage will be limited. Also, note that for Cisco ASA firewalls, you must set the information level to 6 for the log files to include the required attributes.

In addition to checking that the log files have the right attributes, make sure your files and data source meet the following requirements:

  • The data source for the log files must be supported.

  • The format the log files use must match the standard format. When the file is uploaded, app discovery will verify this.

  • The events in the log must have taken place no more than 90 days ago.

  • The log file must include outbound traffic information that can be analyzed for network activity.

Supported vendor firewalls and proxies

App discovery supports the following firewalls and proxies.

Tip: If a data source that you'd like to use is not included here, you can request that it be added to app discovery. To do that, when you're creating a report, select Other for Data source. Then type the name of the data source that you're trying to upload. We'll review the log, and let you know if we add support for that log type.

  • Blue Coat Proxy SG - Access log (W3C)

  • Check Point

  • Cisco ASA Firewall (note that you must set the information level to 6)

  • Cisco IronPort WSA

  • Cisco ScanSafe

  • Cisco Merkai – URLs log

  • Dell Sonicwall

  • Fortiner Fortigate

  • Juniper SRX

  • McAfee Secure Web Gateway

  • Microsoft Forefront Threat Management Gateway (W3C)

  • Palo Alto series Firewall

  • Sophos SG

  • Squid (Common)

  • Squid (Native)

  • Websense - Web Security Solutions - Investigative detail report (CSV)

  • Websense - Web Security Solutions - Internet activity log (CEF)

  • Zscaler

Data attributes for different vendors

Here's the information that is included in log files by different vendors, based to their documentation.

Data source

Target app URL

Target app IP

Username

Origin IP

Total traffic

Uploaded bytes

Blue Coat

Yes

No

Yes

Yes

Yes

Yes

Checkpoint

No

Yes

No

Yes

No

No

Cisco ASA

No

Yes

No

Yes

Yes

No

Cisco FWSM

No

Yes

No

Yes

Yes

No

Cisco Ironport WSA

Yes

Yes

Yes

Yes

Yes

Yes

Cisco ScanSafe

Yes

No

Yes

Yes

Yes

Yes

Dell SonicWall

Yes

Yes

No

Yes

Yes

Yes

Fortigate

No

Yes

No

Yes

Yes

Yes

Juniper SRX

No

Yes

No

Yes

Yes

Yes

McAfee SWG

Yes

No

No

Yes

Yes

Yes

Meraki (Cisco)

Yes

Yes

No

Yes

No

No

MS TMG

Yes

No

Yes

Yes

Yes

Yes

PAN

Yes

Yes

Yes

Yes

Yes

Yes

Sophos

Yes

Yes

Yes

Yes

Yes

No

Websense - Investigative detail report (CSV)

Yes

No

No

Yes

No

No

Websense - Internet activity log (CEF)

Yes

Yes

Yes

Yes

Yes

Yes

Zscaler

Yes

No

Yes

No

Yes

No

Troubleshoot errors when log files are uploaded

When you upload log files, you can track the processing of the logs and see if there were any errors by checking the governance log. If there are errors, check out the information in this guide to help you resolve them.

Error

Description

Resolution

Unsupported file type

The file uploaded is not a valid log file. For example, an image file.

Upload a text, zip, or gzip file that was directly exported from your firewall or proxy.

Internal error

An internal resource failure was detected.

Click Retry to re-run the task.

The log format does not match

The log format you uploaded does not match the expected log format for this data source.

  1. Verify that the log is not corrupt.

  2. Compare and match the log file format to the sample format shown on the upload page.

Transactions are more than 90 days old

All transaction are more than 90 days old and therefore are being ignored.

Export a new log with recent events and re-upload it.

No transactions to catalogue cloud apps

No transaction to any recognized cloud apps are found in the log.

Verify that the log contains outbound traffic information.

Unsupported log type

When you select Data source = Other (unsupported), the log is not parsed. Instead, it is sent for review to the Cloud App Security technical team.

The Cloud App Security technical team builds a dedicated parser for each data source. Most popular data sources are already supported. When an unsupported data source is uploaded, it is reviewed and added to the list of potential new data source parsers.

When a new parser is added to the feature, a notification is included in the Cloud App Security release notes.

Share Facebook Facebook Twitter Twitter Email Email

Was this information helpful?

Great! Any other feedback?

How can we improve it?

Thank you for your feedback!

×