VRRP tests in mixed environment

Motivation

To find a method for replacing a single point of failure created by headnodes in the trial platform. Keepalived already uses VRRP for failover of HAProxy virtual IPs (VIPs) -- can we make use of this?

All you need to know about VRRP (for this task)

Hosts first join the 224.0.0.18 multicast group. VRRP advertisements are send to this address by the Master and received by Backups. If there are multiple hosts sending advertisements, the one with the highest priority is selected as the Master. If all senders have the same priority, the host with the highest IP address is selected as the Master.

There can be multiple VRRP groupings of Masters and backups (called "instances") and each of them is identified by its vrID.

Each advertisement also includes a list of virtual IPs and authentication data (for example secrets shared between hosts being part of the instance).

A sample VRRP packet:

11:04:02.958005 52:54:00:a7:12:5c > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 58: (tos 0xc0, ttl 255, id 98, offset 0, flags [none], proto VRRP (112), length 44)
    172.28.0.136 > 224.0.0.18: vrrp 172.28.0.136 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 150, authtype simple, intvl 1s, length 24, addrs(1): 100.100.100.100 auth "mvDvaixF"

VRRP across servers and switches POC

Because VRRP is an open standard, servers with keepalived installed can send valid VRRP packets that are then interpreted by physical switches.

Here a test setup consisting of a physical L3 switch and 2 virtual machines running on the same host and connected to a L2 bridge was created.

TODO: Add diagram explaining the idea.

HAProxy and keepalived were installed and configured on the VMs using kolla-ansible.

VRRP was configured on the switch to conform to the keepalived config. Here are used commands:

SMIS(config)# router vrrp
SMIS(config-vrrp)# interface vlan 1
SMIS(config-vrrp-if)# vrrp 51 ipv4 100.100.100.100
SMIS(config-vrrp-if)# vrrp 51 priority 1
SMIS(config-vrrp-if)# vrrp 51 text-authentication mvDvaixF
SMIS(config-vrrp-if)# vrrp 51 timer 1

(100.100.100.100 is the HAProxy VIP)

The already configured management VLAN (ID 1) was used here for simplicity:

SMIS(config)# show ip interface vlan 1

vlan1 is up, line protocol is up
Internet Address is 172.28.0.198/16
Broadcast Address  172.28.255.255
IP address allocation method is dynamic
IP address allocation protocol is dhcp

With all the above done, one of the VMs becomes the Master and gets the VIP:

root@ubuntu:~# ip a s ens3
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:a7:12:5c brd ff:ff:ff:ff:ff:ff
    inet 172.28.0.136/16 brd 172.28.255.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet 100.100.100.100/32 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fea7:125c/64 scope link
       valid_lft forever preferred_lft forever

The other server has only its internal IP on corresponding interface and the switch becomes a Backup:

SMIS(config-vrrp-if)# show vrrp detail

vlan1  - vrID 51
 ---------------
  State is Backup
  Virtual IP address is 100.100.100.100
  Virtual MAC address is 00:00:5e:00:01:33
  Master router is 100.100.100.100
 Associated IpAddresses :
 ----------------------
100.100.100.100
  Advertise time is 1 secs
  Current priority is 1
  Configured priority is 1
  Configured Authentication
  Authentication key is mvDvaixF

Stopping keepalived on the Master causes the VIP to switch to the other server and when there are no servers with a VIP, the switch becomes the Master. The moment keepalived goes up on any of the servers, the switch transitions to Backup again and this server becomes the new Master.

Unfortunately, with this setup, the switch doesn't seem to keep track of the internal IP of the current master (as evident on the previous listing where the "Master router is" field shows the VIP) and it doesn't add a route telling how to reach the VIP:

SMIS(config-vrrp-if)# show ip route

S 0.0.0.0/0  [1] via 172.28.0.2
C 172.28.0.0/16 is directly connected, vlan1

Resources

• VRRP Protocol whitepaper http://www2.elo.utfsm.cl/~tel242/exp/04/VRRP_protocol.pdf
• First Hop Redundancy Protocols Configuration Guide, Cisco IOS Release 15M&T https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipapp_fhrp/configuration/15-mt/fhp-15-mt-book/fhp-vrrp.html