Routing with ExpressRoute for Office 365

To properly understand routing traffic to Office 365 using Azure ExpressRoute, you’ll need a firm grasp of the core ExpressRoute routing requirements and the ExpressRoute circuits and routing domains. These lay out the fundamentals for using ExpressRoute that Office 365 customers will rely on.

Some of the key items in the above articles that you’ll need to understand include:

  • ExpressRoute circuits aren’t mapped to specific physical infrastructure, but are a logical connection made at a single peering location by Microsoft and a peering provider on your behalf.

  • There’s a 1:1 mapping between an ExpressRoute circuit and a customer s-key.

  • Each circuit can support up to 3 independent peering relationships (Azure Public peering, Azure Private peering, and Microsoft peering); Office 365 requires Microsoft peering.

  • Each circuit has a fixed bandwidth that is shared across all peering relationships.

  • Any public IPv4 addresses and public AS numbers that will be used for the ExpressRoute circuit must be validated as being owned by you, or assigned exclusively to you by the owner of the address range.

  • ExpressRoute circuits themselves are redundant globally and will follow standard BGP routing practices.

See the FAQ page for more information on services supported, costs, and configuration details. See the ExpressRoute locations article for information on the list of connectivity providers offering Microsoft peering support.

Note: Azure ExpressRoute for Office 365 doesn’t work with IPv6 yet. You’ll need to turn IPv6 off or else the connection will default to it and not use the ExpressRoute connection.

The infrastructure that accepts customer traffic for Office 365 applications are accessible on both the Internet and ExpressRoute, or on multiple ExpressRoute circuits. Network traffic from Office 365 to the customer network will traverse ExpressRoute when both internet and ExpressRoute are available. This introduces the possibility of route asymmetry if traffic from the customer network prefers the internet route. Asymmetrical routes are a problem because devices that perform state-ful packet inspection can block return traffic that follows a different path than the outbound packets followed.

When a customer computer initiates a connection to Office 365 over the internet, the customer endpoints associated with the request must be a publicly routable. The same is true for customer initiated connections over ExpressRoute. With many customers peering directly with Microsoft, having private addresses where duplication is possible between customers isn’t feasible.

The following are scenarios where communications from Office 365 to your on-premises network will be initiated:

For Microsoft to route back to your network for these bi-directional traffic flows, the BGP routes to your on-premises devices must be shared with Microsoft.

To connect to Office 365 over an ExpressRoute circuit, you must configure a peering relationship using the Microsoft peering routing domain. Azure Public and Private peering are not required for Office 365 However, there are services related to Office 365 that require the Azure Public routing domain. For example, Azure RemoteApp and the Office 365 management pack are built on top of Microsoft Azure, as a result of being an application built in Azure, some of the endpoints are available over Azure Public peering.

Other applications such as Office 365 Video, is an Office 365 application; however, Office 365 Video is comprised of three different components, the portal, the streaming service, and the content delivery network. The portal lives within SharePoint Online, the streaming service lives within Azure Media Services, and the content delivery network lives within the Azure CDN. The following table outlines these components.

Component

Underlying application

Routing domain

Use

Office 365 Video portal

SharePoint Online

Microsoft peering

Configuration, upload

Office 365 Video streaming service

Azure Media Services

Azure Public peering

Streaming service, used in the event the video is unavailable from the CDN

Office 365 Video content delivery network

Azure CDN

None

Primary source of video download/streaming. Learn more about Office 365 video networking.

While Azure RemoteApp, the Office 365 management pack, and Office 365 Video are the only applications that have hard dependancies on the Azure Public routing domain, additional applications may require this in the future. To understand which Office 365 features and applications are available, refer to the Office 365 endpoints article. There is a column for each application listed that indicates whether the feature is available using Microsoft peering or not.

Each of the Office 365 features that are available using Microsoft peering are listed in the Office 365 endpoints article by application type and FQDN. The reason for using the FQDN in the tables is to allow customers to manage traffic using PAC files or other proxy configurations. The IP addresses that each of the applications requires are broken up at the application level and not the feature level. For every application that is available over ExpressRoute, the IP address ranges are mentioned in a separate table and the IP ranges that are available over the internet only and both the internet and ExpressRoute are detailed.

These IP addresses are grouped into BGP community tags to make managing routing easier. The community tags in use include:

Community string tag

Applications included

Exchange

Exchange Online

Exchange Online Protection

Skype for Business

Skype for Business Online

SharePoint

SharePoint Online

Other office 365 services

Portal and shared

Authentication and identity

Office Online

Most of the FQDNs listed as being advertised to ExpressRoute and the internet are all inclusive. In some situations we've used a wildcard in a FQDN in a situation where one or more sub-FQDNs are advertised differently than the higher level wildcard FQDN. This usually happens when the wildcard represents a long list of servers that are all advertised to ExpressRoute and the internet and there is a small sub-set of servers or CNAMEs that are only advertised to the internet, or the reverse. Refer to the tables below to understand where the differences are.

This table displays the wildcard FQDNs that are advertised to both the internet and Azure ExpressRoute alongside the sub-FQDNs that are advertised only to the internet.

Wildcard FQDN advertised to ExpressRoute and the internet

Sub-FQDN advertised to the internet only

*.microsoftonline.com

click.email.microsoftonline.com

portal.microsoftonline.com

*.officeapps.live.com

NexusRules.officeapps.live.com

Nexus.officeapps.live.com

odc.officeapps.live.com

odc.officeapps.live.com

cdn.odc.officeapps.live.com

ols.officeapps.live.com

ocsredir.officeapps.live.com

ocws.officeapps.live.com

ocsa.officeapps.live.com

Usually PAC files are intended to send network requests to ExpressRoute advertised endpoints directly to the circuit and all other network requests to your proxy. If you're configuring a PAC file like this, compose your PAC file in the following order:

  1. Include the sub-FQDNs from column two in the above table at the top of your PAC file, sending the traffic towards your proxy.

  2. Include all FQDNs marked advertised to ExpressRoute in this article below the first section, sending the traffic directly to your ExpressRoute circuit.

  3. Include any other network endpoints or rules below these two entries, sending the traffic towards your proxy.

This table displays the wildcard FQDNs that are advertised to the internet only alongside the sub-FQDNs that are advertised Azure ExpressRoute and the internet. For your PAC file above, the FQDNs in column two in the below table are listed as being advertised to ExpressRoute in the link referenced, which means they would be included in the second group of entries in the file.

Wildcard FQDN advertised to the internet only

Sub-FQDN advertised to ExpressRoute and the internet

*.office.com

*.outlook.office.com

home.office.com

portal.office.com

www.office.com

*.office.net

agent.office.net

*.office365.com

outlook.office365.com

smtp.office365.com

*.outlook.com

*.protection.outlook.com

*.mail.protection.outlook.com

*.windows.net

login.windows.net

To route to the Office 365 application of your choosing you’ll need to determine a number of key factors.

  1. How much bandwidth the application will require. Sampling existing usage is the only reliable method for determining this in your organization. Use our calculators only as a starting place.

  2. What egress location(s) you want the network traffic to leave your network from. You should plan to minimize the network latency for connectivity to Office 365 as this will impact performance. Because Skype for Business uses real-time voice and video it is particularly susceptible to poor network latency.

  3. If you want all or a subset of your network locations to leverage ExpressRoute.

  4. What locations your chosen network provider offers ExpressRoute from.

Once you determine the answers to these questions, you can provision an ExpressRoute circuit that meets the bandwidth and location needs. For more network planning assistance, refer to the Office 365 network tuning guide and the case study on how Microsoft handles network performance planning.

Example 1: Single geographic location

This example is a scenario for a fictitious company called Trey Research who has a single geographic location.

Employees at Trey Research are only allowed to connect to the services and websites on the internet that the security department explicitly allows on the pair of outbound proxies that sit between the corporate network and their ISP.

Trey Research plans to use Azure ExpressRoute for Office 365 and recognizes that some traffic such as traffic destined for content delivery networks won’t be able to route over the ExpressRoute for Office 365 connection. Since all traffic already routes to the proxy devices by default, these requests will continue to work as before. After Trey Research determines they can meet the Azure ExpressRoute routing requirements, they proceed to create a circuit, configure routing, and linking the new ExpressRoute circuit to a virtual network. Once the fundamental Azure ExpressRoute configuration is in place, Trey Research adds the Office 365 endpoints supported by ExpressRoute for Office 365 to an automatic proxy configuration (PAC) file or URL to route traffic with customer specific data over the direct ExpressRoute for Office 365 connections.

As shown in the following diagram, Trey Research is able to satisfy the requirement to route Office 365 traffic over the internet and a subset of traffic over ExpressRoute using a combination of routing and outbound proxy configuration changes.

  1. An automatic proxy configuration (PAC) file or URL is used to route traffic through a separate internet egress point for Azure ExpressRoute for Office 365.

  2. Clients are configured with a default route towards Trey Research’s proxies.

In this example scenario, Trey Research is using an outbound proxy device. Similarly, customers who aren’t using Azure ExpressRoute for Office 365 may want to use this technique to route traffic based on the cost of inspecting traffic destined for well-known high volume endpoints.

The highest volume FQDNs for Exchange Online, SharePoint Online, and Skype for Business Online are the following:

ExpressRoute customer edge network
  • outlook.office365.com

  • outlook.office.com

  • <tenant-name>.sharepoint.com

  • <tenant-name>-my.sharepoint.com

  • <tenant-name>-<app>.sharepoint.com

  • *.Lync.com

Learn more about deploying and managing proxy settings in Windows 8 and ensuring Office 365 isn't throttled by your proxy.

With a single ExpressRoute circuit, there is no high availability for Trey Research. In the event Trey’s redundant pair of edge devices that are servicing the ExpressRoute connectivity fail, there is not an additional ExpressRoute circuit to failover to. This leaves Trey Research in a predicament as failing over to the internet will require manual re-configuration and in some cases new IP addresses. If Trey wants to add high availability, the simplest solution is to add additional ExpressRoute circuits.

The last scenario, Routing Office 365 traffic over ExpressRoute is the foundation for even more complex routing architecture. Regardless of the number of locations, number of continents where those locations exist, number of ExpressRoute circuits, and so on, being able to route some traffic to the internet and some traffic over ExpressRoute will be required.

The additional questions that must be answered for customers with multiple locations in multiple geographies include:

  1. Do they require an ExpressRoute circuit in every location? If using Skype for Business or the customer is concerned with latency sensitivity for SharePoint Online or Exchange Online, an ExpressRoute circuit is recommended in each continent where the customer has offices. See the Skype for Business media quality and network connectivity guide for more details.

  2. If an ExpressRoute circuit isn’t available in a particular region, how should Office 365 destined traffic be routed?

  3. What is the preferred method for consolidating traffic in the case of networks with many small locations?

Each of these presents a unique challenge that requires you to evaluate your own network as well as the options available from Microsoft.

Consideration

Network components to evaluate

Circuits in more than one location

Cost, latency, and bandwidth needs must be compared.

Use BGP route cost, PAC files, and NAT to manage routing with multiple circuits.

Routing from locations without an ExpressRoute circuit

DNS forwarding can be used to allow remote offices to discover the appropriate endpoint.

Clients in the remote office must have a route available that provides access to the ExpressRoute circuit.

Small office consolidation

Available bandwidth and data usage should be carefully compared.

Note: Microsoft will prefer ExpressRoute over the internet if the route is available regardless of physical location.

Each of these considerations must be taken into account for each unique network. Below is an example.

Example 2: Multi-geographic locations

This example is a scenario for a fictitious company called Humongous Insurance who has multiple geographic locations.

Humongous Insurance is geographically dispersed with offices all over the world, who will implement Azure ExpressRoute for Office 365 to keep the majority of their Office 365 traffic on direct network connections. Humongous Insurance also has offices on two additional continents. The employees in the remote office where ExpressRoute is not feasible will need to route back to one of the two primary facilities to use an ExpressRoute connection.

The guiding principle is to get Office 365 destined traffic to a Microsoft datacenter as quickly as possible. In this example, Humongous Insurance must decide if their remote offices should route over the internet to get to a Microsoft datacenter over any connection as quickly as possible or if their remote offices should route over an internal network to get to a Microsoft datacenter over an ExpressRoute connection as quickly as possible.

Microsoft’s datacenters, networks, and application architecture are designed to take globally disparate communications and service them in the most efficient way possible. Requests destined for Office 365 that remain on customer networks longer than necessary won’t be able to take advantage of this architecture.

In Humongous Insurance’s situation, they should proceed depending on the applications they intend to use over ExpressRoute. For example, if they’re a Skype for Business Online customer, or plan to leverage ExpressRoute connectivity when connecting to external Skype for Business Online meetings, the design recommended in the Skype for Business Online media quality and network connectivity guide is to provision an additional ExpressRoute circuit for the third location. This may be more expensive from a networking perspective; however, routing requests from one continent to another before delivering to a Microsoft datacenter may cause a poor or unusable experience during Skype for Business Online meetings and communications.

If Humongous Insurance isn’t using or doesn’t plan to leverage Skype for Business Online in any way, routing Office 365 destined network traffic back to a continent with an ExpressRoute connection may be feasible. In both cases, routing internet destined traffic to the internet at the local site is recommended to take advantage of the content delivery networks that Office 365 relies on.

ExpressRoute multi-geography

When Humongous Insurance is planning their multi-geography strategy, there are a number of things to consider around size of circuit (discussed here), number of circuits, failover, and so on.

With ExpressRoute in a single location with multiple regions attempting to use the circuit, Humongous Insurance wants to ensure that connections to/from Office 365 from the remote office are sent to the Office 365 datacenter nearest headquarters and received by the headquarters location. To do this, Humongous Insurance implements DNS forwarding to reduce the number of round trips and DNS lookups required to establish the appropriate connection with the Office 365 environment closest to the headquarters internet egress point. You can also learn to Assign a Conditional Forwarder for a Domain Name.

In this scenario, traffic from the remote office would resolve the Office 365 front end infrastructure in North America and leverage Office 365 to connect to the backend servers according to the architecture of the Office 365 application. For example, Exchange Online would terminate the connection in North America and those front end servers would connect to the backend mailbox server wherever the tenant resided. See Client connectivity for more information about how each application is architected to handle customer connectivity.

If Humongous has major offices in multiple continents, a minimum of one circuit per continent is recommended in order to reduce latency for sensitive applications such as Skype for Business. If all offices are in a single continent, or is not using real time collaboration, having a consolidated or distributed egress point is a customer specific decision that would depend on the number of people per location, the application usage at each location, high availability requirements, and so on. When multiple circuits are available, BGP routing will ensure failover should any single circuit become unavailable.

Learn more about sample routing configurations and https://azure.microsoft.com/en-us/documentation/articles/expressroute-config-samples-nat/.

Selective routing with ExpressRoute may be needed for a variety of reasons, such as testing, rolling out ExpressRoute to a subset of users. There are various tools customers can use to selectively route Office 365 network traffic over ExpressRoute:

  1. Route filtering/segregation – allowing the BGP routes to Office 365 over ExpressRoute to a subset of subnets or routers. This selectively routes by customer network segment or physical office location. This is common for staggering rollout of ExpressRoute for Office 365.

  2. PAC files/URLs – directing Office 365 destined network traffic for specific FQDNs to route on a specific path. This selectively routes by client computer as identified by PAC file deployment.

  3. BGP communities – filtering based on BGP community tags allows a customer to determine which Office 365 applications will traverse ExpressRoute and which will traverse the internet.

    Note: BGP communities will be released for ExpressRoute in the near future.

See Also

Network connectivity to Office 365

Azure ExpressRoute for Office 365

Network planning with ExpressRoute for Office 365

Implementing ExpressRoute for Office 365

Media Quality and Network Connectivity Performance in Skype for Business Online

Optimizing your network for Skype for Business Online

ExpressRoute and QoS in Skype for Business Online

Call flow using ExpressRoute

Using BGP communities in ExpressRoute for Office 365 scenarios (preview)

Office 365 performance tuning using baselines and performance history

Performance troubleshooting plan for Office 365

Office 365 URLs and IP address ranges

Office 365 network and performance tuning

Share Facebook Facebook Twitter Twitter Email Email

Was this information helpful?

Great! Any other feedback?

How can we improve it?

Thank you for your feedback!

×