I’ve been involved in a few discussion with people at Microsoft and it seems there has been a swing in sentiment with regards to whether customers should use Microsoft Peering over Express route at all. The long and short of Microsoft Peering is that it enables several Azure services to be available over Express route rather than the general internet. This includes Exchange Online, SharePoint Online, Skype for Business, and Dynamics 36. It excludes services such as Azure websites and Azure storage as these are available over ‘Public Peering’ instead.
When we first deployed Office365 (probably a good 4 years ago now I think?) we saw a perceivable increase in performance when running over Express Route rather than the internet – Our PA’s were running Outlook in the non-cached mode due to the number of mailboxes and calendars they were looking after. Express route made a big difference here.
It’s hard to say at this point why the recommendation would have changed from a reliable path (Express route) to big bad internet routing. To be fair, Microsoft would probably peer with most major ISPs in some way or another so maybe that’s where they are coming from (that it’ll effectively be the same number of hops there anyway?). It’s still a contended, and non QOS supporting path.
I see some murmurings on the internet that they are looking to combine the Public and Microsoft peering into a single routing domain. I’d welcome this as it means a few less BGP peering points. All of the prefixes they advertise now have BGP communities so it’s easy enough to distinguish different services being advertise over the same peering.
Microsoft recently announced that it will be possible to connect to Office 365 services via an express route connection. Leveraging an express route connection, which in most cases will offer faster speeds than your internet link, will mean a faster and more optimal flow through to this azure service. Additionally, many other azure public services have already been made available over express route by creating a special ‘public peering’. In my case, the on-prem storsimple appliance was continually using all of our internet bandwidth while doing site snapshots and this necessitated setting this up.
Simply, by setting up an azure public peering express route connection you will have two paths to azure services.
Two paths to express route services
The documentation around how this is achieved is fairly sparse and I had some questions around how this would be achieved technically. Following on from my trial and error I have the following information which has been proven in our environment.
- What routes are advertised from a Public Express Route BGP peer?
Upon creating an adjacency you will receive (as of writing) 128 prefixes. These ranges from /17’s through to a couple of /29’s. If you receive a default route on your internet link, these prefixes will be more specific and therefore should be the new chosen path.
- How should we NAT egress traffic?
I have experimented with this a little, and have natted the traffic both behind the same addresses as our internet connection uses, and also unique addresses. I was worried that using the same PI space would cause asynchronous routing, with traffic leaving via our express route circuit, and returning via our internet (because our internet AS advertises that range). It seems however that Microsoft must mark this traffic somehow (MPLS?) as traffic returns to the same path it came from, even if the source address is the same. This is handy if you have a firewall that is natting all the traffic, it means you don’t need to change the NAT dependent on the path it takes.
This post is a work in progress and will be updated as and when new information comes to light.