These next couple of posts will focus on the different load balancing mechanisms that are available in Microsoft Azure. Going from Azure Load Balancer, Application Gateway, Traffic Manager, Front Door, and how these compare with other NVA’s that are available on the market. This first post will focus on Azure Load Balancer. Then I will focus on how to combine the different services depending on use-case and design.
What is Azure Load Balancer?
Azure Load Balancer is a PaaS service to allow load balancing of traffic to virtual machines running in Azure or for outbound connections using SNAT. It does not support load balancing of traffic to PaaS services, and another important note is that Azure Load Balancer operates at OSI level 4 for TCP and UDP meaning that there is not intelligent load balancing mechanisms for routing traffic based upon URL or HTTP Content. Ulike other Load Balancers, Azure LB does not terminate, respond, or otherwise interact with the payload of a UDP or TCP flow. It is not like a reverse proxy, where TCP connections for instance a terminated and reestablished for multiplexing. However this means that other protocols such as ICMP will not function and you would therefore require a PIP directly on the Instance to allow this to work.
Azure Load Balancer consists of 5 objects.
- Public IP Address (or Private IP, depending if it is intended to be public facing or internal)
- Backend Pool (Which virtual machines traffic should be sent to)
- Health Probes (Rules to detect health of the resources in the backend pool)
- Load Balancing Rule (Combines all the objects above together with rules on how traffic should be load balanced to the backend resources in the backend pool and on which backend port.
- Optional NAT Rule: Allows Port NAT (Address translation) to one of the backend servers in the pool on a specific port. This can be for instance Port NAT to 3389.
Azure Load Balancer comes in two flavors, Basic and Standard which have some differences in terms of functionality, availability and pricing.
Some highlights, is the lack of SLA support for Basic SKU and support for Availabilty Zones.
- Standard SKU Support HTTPS Health Probes which is essentially an TCP probe and issue an HTTP GET with the specified path (and if Health Probes are down, all TCP flows continue for Standard and for Basic it will just terminate) NOTE: HTTPS probes do not support mutual authentication with a client certificate.
- Standard SKU Also supports a feature called HA Ports which allows for load balancing of multiple ports at the same time, which is useful if you have NVA (Network Virtual Appliances) such as Firewalls where you use them to filter outbound traffic on a range of port ranges. Only issue now is that HA Ports is only supported for internal load balancers and not public facing. NOTE: If you are using an NVA ensure that the vendor supports the use of HA Ports in Azure (Such as Barracuda: https://github.com/barracudanetworks/ngf-azure-templates )
- Standard SKU also supports TCP Reset, which is a mechanism to Endpoints receiving TCP RST packets close the corresponding socket immediately.
Outbound Connections
Something that is equally important as load balancing traffic inbound to virtual resources is also for outbound connections from resources in Azure, especially if you intend to have certain services whitelisted for communication with other 3.party services, Azure LB can be a service that can be used to control outbound communication flow as well.
VM without Public IP
If you have a VM without any public IP, when the VM creates an outbound flow, Azure translates the private source IP address of the outbound flow to a public source IP address, using SNAT with port masquerading. The public IP address used for this outbound flow is not configurable and does not count against the subscription’s public IP resource limit. This public IP address does not belong to you and cannot be reserved. If you redeploy the VM or Availability Set or virtual machine scale set, this public IP address will be released and a new public IP address gets requested. This is not something can be used if you intend to whitelist traffic coming from certain IP ranges. Of course if you have many virtual machines in the same VNET which needs to communicate outbound you can hit a limit quite fast.
VM With Public IP
When a Public IP address is used, the VM uses the Public IP address for all outbound flows. A public IP assigned to a VM is a 1:1 relationship (rather than 1: many) and implemented as a stateless 1:1 NAT, so therefore no port masquerading.
VM with Public facing Load Balancer
When a public Load Balancer resource is associated with VM instances, each outbound connection source is rewritten. The source is rewritten from the virtual network private IP address space to the frontend Public IP address of the load balancer. If you are getting issues with SNAT port exhaustion you can assign multiple frontend IP addresses for outbound connections and multiply the number of SNAT ports available. Create a frontend IP configuration, rule, and backend pool to trigger the programming of SNAT to the public IP of the frontend. NOTE: The rule does not need to function and a health probe does not need to succeed.
Now in the case of a scenario where you have a Virtual machine that has both a Public IP and part of a Azure LB Public facing. The first scenario will take priority for outbound SNAT connections.
Now for Standard based SKU you also have the ability to control outbound SNAT. Standard Load Balancer uses all candidates for outbound flows at the same time when multiple (public) IP frontends is present, but using ARM you can disable SNAT.
"loadBalancingRules": [ { "disableOutboundSnat": false } ]
Health Probes
When defing health probes you are defining how Azure Load Balancer can check the virtual resources in the backend pool if they are healthy or not. Targets that do not respond to the health probe will be taken out of the backend pool to ensure that traffic is going only to healthy backend clients.
All Health Probe traffic will be originating from the virtual IP address that Azure controls, IP:168.63.129.16. Ensure that this IP address can communicate with the backend pool resources if not health probes will fail and therefore Azure will mark the service as down. Should also be noted that this VIP is running with IIS 10 / WS 2016 as source.
NOTE: No enduser traffic will be originating from this IP address, it will be sources directly to the client.
This virtual public IP address also facilitates the following things:
- Enables the VM Agent to communicate with the Azure platform to signal that it is in a “Ready” state.
- Enables communication with the DNS virtual server to provide filtered name resolution to the resources (such as VM) that do not have a custom DNS server. This filtering makes sure that customers can resolve only the hostnames of their resources.
- Enables the VM to obtain a dynamic IP address from the DHCP service in Azure.
- Enables Guest Agent heartbeat messages for the PaaS role.
If you still need confirmation about if this IP is safe, you can read more about it here –> https://docs.microsoft.com/en-us/azure/virtual-network/what-is-ip-address-168-63-129-16
Limitations:
- A standalone virtual machine resource, availability set resource, or virtual machine scale set resource can reference one SKU, so a virtual machine associated with a Basic LB cannot be referenced in a Standard LB
- A Load Balancer rule cannot span two virtual networks, this also applies for VNET Peering. Frontends and their related backend instances must be located in the same virtual network.
FAQ:
- Can we migrate from one SKU to another? No there is no direct migration option, you will need to restablish a new Load Balancer with a new IP address and then move the rules across. Microsoft is working on an option to migrate from basic to standard IP SKU but it is on the backlog
- Can we use HA Ports for public facing LB services? No, this is not a supported feature yet, again something that Microsoft is working on.
- Can we define weights on backend resources? No this is not a supported feature as part of LB
- Can we define load balancing methods using Azure LB? No, Azure LB does round robin load balancing for new incoming TCP connections, not for new incoming HTTP requests. And this cannot be changed
- Can we define autoscaling of resources based upon metric in Azure LB? You can connect it to a Action Rule using Azure Monitor to trigger an job, or it can be done using VM Scale Sets.
- Can we define a threshold on bandwidth? No this is not supported.
- Can we customize health probes to send specific payload? No for HTTP based you can only define URL Path and it will only check if you get a HTTP 200 code back.
