If you get an error when trying to open a protected file in Outlook 2016 for Mac, it's probably because the Active Directory Rights Management Services (AD RMS) mobile device extension hasn't been installed on your organization’s existing AD RMS deployment.
What is a protected file?
A protected file is a file which has its data encrypted by Rights Management Services (RMS) in order to prevent sensitive information from being printed, forwarded, or copied by unauthorized people. Because of this, a protected file can’t be accessed without a license.
What is AD RMS?
AD RMS lets mobile and Mac users protect and consume sensitive data, including protected text files in .txt, .csv, and .xml formats; protected image files in .jpg, .gif, and .tif formats; and Office files in the .ppdf format.
You'll need to ask your Office 365 administrator or IT department to download and deploy the AD RMS extension. Include this link to the Active Directory Rights Management Services Mobile Device Extension article in your request.