Today I discovered a very simple way to get VLANs talking to bonded interfaces while I was working on a Parallels CloudLinux Cluster. The hardware was a bunch of Dell R620’s and a PowerConnect 6248 Switch. Its significant to note because it took me all day to realize that most of the issues I was seeing were in fact switch related and luckily for me I had already documented that aspect of Network Configuration in a much earlier post regarding an M6220 which has a similar command structure.
Basically the switch ports need to be bonded as a LAG group and then configured to trunk the required VLAN’s at the port-channel level. The basic Config looks like:
interface port-channel 1 description ‘LAG-1′ hashing-mode 1 switchport mode trunk switchport trunk allowed vlan add 10,20,30,40,50 interface ethernet 1/g1 channel-group 1 mode auto description ‘LACP Cluster Trunk 1′ switchport mode trunk switchport trunk allowed vlan add 10,20,30,40,50 interface ethernet 1/g2 channel-group 1 mode auto description ‘LACP Cluster Trunk 1′ switchport mode trunk switchport trunk allowed vlan add 10,20,30,40,50 exit
We are using port 1 and 2 on the switch and trunking 5 VLAN’s to it.
Linux Network Bonding
First an overview of the Bonding Setup, in particular the VLAN configuration. To bond Ethernet interfaces you need (at a minimum), two interfaces. Their configuration is almost identical other than the device field and unlike a normal static interface they have no IP address defined and nominate a “MASTER” interface instead. Here is a working example, located in /etc/sysconfig/network-scripts:
File: /etc/sysconfig/network-scripts/ifcfg-eth2 DEVICE=eth2 ONBOOT=yes BOOTPROTO=none USERCTL=no MASTER=bond0 SLAVE=yes NM_CONTROLLED=no File: /etc/sysconfig/network-scripts/ifcfg-eth3 DEVICE=eth3 ONBOOT=yes BOOTPROTO=none USERCTL=no MASTER=bond0 SLAVE=yes NM_CONTROLLED=no File: /etc/sysconfig/network-scripts/ifcfg-bond0 DEVICE=bond0 IPADDR=192.168.0.10 PREFIX=24 ONBOOT=yes BOOTPROTO=none NM_CONTROLLED=no USERCTL=no BONDING_OPTS="mode=2 miimon=100 updelay=300 downdelay=300"
You also need to have the bonding kernel module loaded, this is done by creating a file called bonding.conf in /etc/modprobe.d with a single line:
alias bond0 bonding
Lets say we have 5 VLANs we need to bring up:
- A backup VLAN (VLAN-10)
- A Management VLAN (VLAN-20)
- A cluster VLAN (VLAN-30)
- A private Network for HA (VLAN-40)
- A DMZ VLANs (VLAN-50)
And we use 192.168.10.0/24 as VLAN-10 and 192.168.20.0/24 as VLAN-20 etc
To assign VLAN-10 traffic to our LACP Trunk we create an ifcfg-bond0.10 file by first copying the bond0 file, changing the device line, the IP and adding a new tag called VLAN=yes. Here is what the /etc/sysconfig/network-scripts/ifcfg-bond0.10 file looks like:
File: /etc/sysconfig/networking-scripts/ifcfg-bond0 DEVICE=bond0.10 IPADDR=192.168.10.10 PREFIX=24 ONBOOT=yes BOOTPROTO=none USERCTL=no BONDING_OPTS="mode=2 miimon=100 updelay=300 downdelay=300" VLAN=yes
Bringing up the interfaces
We can start each VLAN interface manually if we wish, once bond0 is up the rest can be done in any order using:
The rest of the files will look the same, but the name will have the VLAN identifier appended and internally the DEVICE and IPADDR fields will be set accordingly.