I’ve been involved in a few discussion with people at Microsoft and it seems there has been a swing in sentiment with regards to whether customers should use Microsoft Peering over Express route at all. The long and short of Microsoft Peering is that it enables several Azure services to be available over Express route rather than the general internet. This includes Exchange Online, SharePoint Online, Skype for Business, and Dynamics 36. It excludes services such as Azure websites and Azure storage as these are available over ‘Public Peering’ instead.
When we first deployed Office365 (probably a good 4 years ago now I think?) we saw a perceivable increase in performance when running over Express Route rather than the internet – Our PA’s were running Outlook in the non-cached mode due to the number of mailboxes and calendars they were looking after. Express route made a big difference here.
It’s hard to say at this point why the recommendation would have changed from a reliable path (Express route) to big bad internet routing. To be fair, Microsoft would probably peer with most major ISPs in some way or another so maybe that’s where they are coming from (that it’ll effectively be the same number of hops there anyway?). It’s still a contended, and non QOS supporting path.
I see some murmurings on the internet that they are looking to combine the Public and Microsoft peering into a single routing domain. I’d welcome this as it means a few less BGP peering points. All of the prefixes they advertise now have BGP communities so it’s easy enough to distinguish different services being advertise over the same peering.
The benefits of Express route are well documented, and from the horses mouth, “ExpressRoute connections offer higher security, reliability, and speeds, with lower and consistent latencies than typical connections over the Internet”. A point of contention has been this so called reliability.
The only way to guarantee reliability in packet switched networks is QOS. Sure, through capacity management and or ‘massive pipes’ this can be avoided to a certain extent, but there is always a risk that one device/user/application could consume all of that bandwidth leaving other systems with a smaller piece of the pie.
We have recently adopted Skype for Business PSTN Calling and through this have developed a need for a priority queue on EF (voice traffic effectively) marked packets. Microsoft’s documentation is quite clear on how it thinks QOS should be deployed.
It needs to be deployed end-to-end.
A QoS capable connection must be configured end-to-end (PC, network switches and routers to the cloud) as any part in the path that fails to support QoS could degrade the quality of the entire call
Your express route provider must provide a class of service for EF packets.
Each ExpressRoute network service provider will have a class of service (QoS) that is appropriate for real-time voice and video. This COS is called ‘Expedited Forwarding’ (EF) for voice and ‘Assured Forwarding’ (AF) for video
The problem is that our network provider and Microsoft doesn’t mandate this requirement despite it’s clear stance on it. I’ve labored through support tickets both with our provider and Microsoft – The provider has no interest as there is no pressure (from Microsoft, or other customers at this point) and Microsoft won’t force them to implement it.
The post serves little more purpose than to say, ‘be careful’ when expecting your express route provider to support QOS. They may not, and they are not required to.
We are looking at alternative network providers.
As users of Azure RemoteApp I was tasked with finding a replacement due to the product being retired.
We used the service primarily as a remote access solution, allowing users to RDP to their desktops (assuming they were turned on).
The preferred solution recommended by Azure is Citrix Xenapp Express, a cloud only version of XenApp. This was trialed internally and vetoed almost straight away after deployment as it lacked two factor authentication – this is a huge oversight in my opinion. While views on this will vary company to company, I personally feel that ALL external applications should be protected by two factor authentication. It’s simply not good enough to rely on a username and password.
I decided to take a look at the Netscaler VPX appliance. It is available in the Azure Microsoft and can be deployed to any ARM environments.
- Clientless access (although you need to have the Microsoft terminal services client installed – native in Windows, you need to install it on MacOSX, Android etc)
- Works on every device I’ve tried – Includes iPhone, Android Tablets, MacOSX etc
- Much cheaper than RemoteApp. A Netscaler VPX enterprise license (which you need for RDP Proxy) and 200mb throughput will be ~£8000 per year + Azure VM costs. We were spending circa £15,000 for 200 or so users.
- Good user experience – web portal is fairly straight forward.
- Performant – test users report their desktops are much more responsive than publishing RDP over Remote App.
- Feature Rich – supports web bookmarks, load balancing, DDoS protection
- I wouldn’t say Netscaler VPX is very intuitive to administer/configure. I think it’s more than just a lack of familiarity with the system.
- IaaS – you need to maintain the system, patch it, and make sure it’s configured correctly
- Supports any authentication methods you can think of. We are using radius in conjunction with the Microsoft multi factor authentication server.
The system is being trialed by our I.T department but it’s so far proved to be really good.
The Azure Application Gateway
Azure have released an application gateway with some WAF functionality which “protects web applications from common web-based attacks like SQL injection, cross-site scripting attacks, and session hijacks”. I deployed Barracuda WAF’s in our Azure architecture several years ago but subsequently got rid of them – they are expensive solution if you don’t have a lot of applications behind them. We were paying somewhere in the region of £10,000 per year per device. The costs add up when you need 2 for a high availability set and then another set in a different geo-region. £40,000 to protect a handful of websites. The application gateway is attractive from a cost perspective, although the WAF pricing itself hasn’t been confirmed as it’s still in preview.
The pricing is very attractive, although the WAF functionality is to ‘TBC’
I’m keen to evaluate this service and would like to find out:
- How much watering and feeding will they require? WAF’s like most security devices have an administrative overhead.
- How granular are the WAF rules – can the rules be tweaked? Will entering Irish names trigger SQL injection rules which can’t be altered?
- Would we be better off using Cloudflare in front of our websites?
- Does it play nice with authentication? (namely ADFS / Azure AD Authentication)
- What is it’s logging capability like, can we export it to syslog server?
I’ll do another post when I have had a good look around the product. It’s feature set overlaps rather confusingly with the ‘Azure AD Proxy’ apart from the WAF features – I’m not sure if there is a use case for both of the products still.
A while ago Azure announced a new feature allowing you to connect VNets together without express route or VPN.
VNet peering is a mechanism that connects two virtual networks (VNets) in the same region through the Azure backbone network. Once peered, the two virtual networks appear as one for all connectivity purposes. They are still managed as separate resources, but virtual machines in these virtual networks can communicate with each other directly by using private IP addresses.
The product page can be found here.
The Existing Architecture
Originally, your architecture may have consisted of several VNets and all of these would have had a gateway (a billable item) and a form of connectivity between them – either express route or VPN. There are some inefficiencies here as all traffic between VNets needs to be routed via your express route circuit. Bandwidth and the actual location of your express route termination points may be problems here. A 1000mb/s link may be plenty between your on-prem networks and Azure, but is it enough for your traffic between all of your VNets? With the below architecture we have 4 gateways for 4 VNets.
Legacy Azure Architecture – A gateway in each VNet
The New Architecture
VNet peering does away with the requirement to have a gateway on every VNet. We can create a VNet which serves as the link between your on-prem networks and have the rest connected via VNet peering. This is even supported between VNets that are in different subscriptions. To use the same reference architecture but changed to adopt VNet peering, we will have something like the below.
VNet Peering Azure Architecture – One more VNet but 3 fewer gateways. Profit!
So we have one extra VNet, but 3 fewer gateways. The gateways are what cost money in this setup, so this will not only be a cheaper setup, but a more efficient one. All traffic between VNets will be via the (blazing fast!!!) azure backbone and not your express route link. At the moment VNet peering traffic is free of charge, so you will save some money on your ingress/egress express route charges.
As with everything Azure, please do your own research as pricing and features change regularily.