In my previous blog, we discussed about:
- What is Beacon Probing?
- Why do we need it?
- How it is different from Link Status Only Failover Detection Method?
- How Beacon Probing works?
I would highly recommend you to read my previous blog on Beacon Probing before reading further. CLICK HERE to read Part-1 of this blog series.
In this article, we’ll be discussing about VMware recommendations or Best Practices for enabling/configuring Beacon Probing in vSphere Environment
BEST PRACTICE NO-1
- VMware recommends having 3 or more physical NICs in a team for beaconing to work properly. If you have only 2 uplinks in a team, ESXi host will not be able to correctly determine which uplink has gone down.
Let’s analyze below scenario to understand WHY?
In this scenario, upstream switch connectivity between Access and Core switch is down. Beacon packets sent out by vmnic1 will be dropped at Access switch level due to link failure. Due to beacon drop at Access Switch level, Core Switch will never be able to forward the beacon packet to its peer device. Hence, NO beacon of vmnic1 will be received by vmnic0 and VMkernel will not be able to figure it out the exact status of vmnic1 adapter.
Similarly, beacon probes sent out by vmnic0 will by dropped at Core Switch Level. Due to which vmnic1 doesn’t receive any beacon probes for vmnic0. Again, VMkernel will not have any clue about the exact status of vmnic0 adapter.
In such scenarios, ESXi will sent out the VM traffic to both the uplinks and hoping that traffic will go through one of the uplink. This behavior in vSphere Networking is known as “SHOTGUN EFFECT”
To avoid Shotgun effect, VMware recommends to use 3 or more uplinks in a team for beacon probing.
BEST PRACTICE NO-2
- VMware doesn’t recommend multiple Physical NIC connection to the same physical switch. This can cause beacon probing to fail to detect issues beyond the switch.
Below mentioned topologies are not supported:
In this topology:
- If link between pSwitch0 and DpSwitch0 goes down, beacon sent out by pNic0 will detect failure at DpSwitch0 level and beacons will be retransmitted to pSwitch0 and received by pNiC1. Since ESXi kernel receives beacons for both the NICs as expected so it will mark Uplink state for both the uplinks as UP which is not correct. VM traffic forwarded to this NIC will be dropped at upstream switch. (DpSwitch0 in our case).
In this Topology also:
If link between DpSwitch and CorepSwitch goes down, beacon packet will transmitted back to pSwitch and will reach out to all the uplinks. VMkernel will shows state of each Uplink as UP because beacon probes were received for each uplink from vmkernel perspective. ESXi Kernel will not be able to detect uplink switch failures even though beacon probing is enabled.
BEST PRACTICE NO-3
- IP Hash Load Balancing is not supported with Beacon Probing enabled. One of the pre-requisite of IP Hash Based Load Balancing is configuring Etherchannel or LACP on physical uplinks. Physical Switch refers all uplinks configured in Etherchannel as single logical uplink.
In this scenario, when pNIC0 sends a beacon to check on pNIC1, packet will never reach pNIC1. Because from physical switch perspective, pNIC0, pNIC1, pNIC2,pNIC3 are same uplink port & switch never forward the packet back to the sender port. So pNIC1 never receives beacon probes from pNIC0. Same goes for other uplinks as well. Since ESXi kernel doesn’t receive beacon probes for any of the uplinks. It can’t determine each pNIC status correctly even though beacon probing is enabled.
That’s the reason, Beacon Probing is not supported with IP Hash based load balancing. Link Status Only algorithm needs to be used with IP Hash based load balancing.
WHEN SHOULD YOU ENABLE BEACON PROBING?
Default failover detection method is “Link Status Only”. Beacon probing dumps a lot of L2 traffic onto the wire which increases overhead on network and consumed more CPU cycles of vmkernel. You must enable beacon probing when upstream link failures may impact VM availability and there is no “Link State Tracking” support on the physical switch.
I hope you guys might have found this article useful. Your feedback is very much valuable for me so please share your feedback and queries in comment box. Your feedback will keep motivating me to write more and more articles in future.
Keep sharing Keep Learning!!