Azure Express Route Public Peering

Microsoft recently announced that it will be possible to connect to Office 365 services via an express route connection. Leveraging an express route connection, which in most cases will offer faster speeds than your internet link, will mean a faster and more optimal flow through to this azure service. Additionally, many other azure public services have already been made available over express route by creating a special ‘public peering’. In my¬†case, the on-prem storsimple appliance was continually using all of our internet bandwidth while doing site snapshots and this necessitated setting this up.

Simply, by setting up an azure public peering express route connection you will have two paths to azure services.

Azure-Express-Route-Public-Peering - New Page

Two paths to express route services

 

The documentation around how this is achieved is fairly sparse and I had some questions around how this would be achieved technically. Following on from my trial and error I have the following information which has been proven in our environment.

  • What routes are advertised from a Public Express Route BGP peer?
    Upon creating an adjacency you will receive (as of writing) 128 prefixes. These ranges from /17’s through to a couple of /29’s. If you receive a default route on your internet link, these prefixes will be more specific and therefore should be the new chosen path.
  • How should we NAT egress traffic?
    I have experimented with this a little, and have natted the traffic both behind the same addresses as our internet connection uses, and also unique addresses. I was worried that using the same PI space would cause asynchronous routing, with traffic leaving via our express route circuit, and returning via our internet (because our internet AS advertises that range). It seems however that Microsoft must mark this traffic somehow (MPLS?) as traffic returns to the same path it came from, even if the source address is the same. This is handy if you have a firewall that is natting all the traffic, it means you don’t need to change the NAT dependent on the path it takes.

This post is a work in progress and will be updated as and when new information comes to light.

4 comments

  1. Hi Stew,

    I wanted to ask if you have any update on this. I have stood up my bgp (Microsoft & Public) with azure but am seeing asymmetric routing as you described above. I am looking at raising a ticket tomorrow in the portal but came across your post so wanted to ask if you got azure to mark/tag the return traffic.

    regards,
    Jennings

    1. Hi Jennings,

      To be honest this has ‘just worked’ for us. Traffic is natted somewhere in the path to Azure as I can see a static public IP address that I’m not familiar in the logs of cloud services we access. I’m not sure whether the NAT happens at our express route network service provider or at Microsofts end.

      The only caveat I’ve come across with this is setting up a site to site vpn – I was trying to setup an ipsec tunnel to a new subscription and found this was routing asymmetrically over express route. The caveat is documented here.

      Static route requirement: If your local network is connected to both ExpressRoute and a Site-to-Site VPN, you must have a static route configured in your local network to route the Site-to-Site VPN connection to the public Internet.

  2. Hi Stew,

    Thanks for your reply. I came across that caveat just last night and which is very similar to my setup. I have a site to site with Azure to access our domain controller and also the BGP peering for 0365 and Azure VM. What I found is that when the BGP sessions were up, the ipsec tunnel wouldn’t connect. It wasn’t until a static route was entered in our edge device that the ipsec tunnel came up. I have now gone with that setup as per the above caveat.

    Thanks your getting back Stew.

    cheers

    1. No worries, glad you’ve got it sorted. It’s a bit of a pain having to use those static routes as it’s difficult to setup resiliency for them – our internet is multihomed across multiple providers so the only way we can have this static route failover is to use IP SLA to detect failure of the primary path. It’s a lot more clunky that letting BGP local pref / AS prepend do it’s thing!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.