Navigation

Tags Archives: vlan

LPIC3 DIPLOMA Linux Clustering – LAB NOTES: Lesson VLANs

By on May 11, 2021 in

 

LAB on VLANs

 

These are my notes made during my lab practical as part of my LPIC3 Diploma course in Linux Clustering. They are in “rough format”, presented as they were written.

 

 

LPIC3 Syllabus for VLANs

 

364.4 Network High Availability
Weight: 5
Description: Candidates should be able to configure redundant networking connections and manage VLANs.

Furthermore, candidates should have a basic understanding of BGP.

Key Knowledge Areas:
• Understand and configure bonding network interface
• Network bond modes and algorithms (active-backup, blance-tlb, balance-alb,
802.3ad, balance-rr, balance-xor, broadcast)
• Configure switch configuration for high availability, including RSTP
• Configure VLANs on regular and bonded network interfaces
• Persist bonding and VLAN configuration
• Understand the principle of autonomous systems and BGP to manage external
redundant uplinks
• Awareness of traffic shaping and control capabilities of Linux
 

Partial list of the used files, terms and utilities:
• bonding.ko (including relevant module options)
• /etc/network/interfaces
• /etc/sysconfig/networking-scripts/ifcfg-*
• /etc/systemd/network/*.network
• /etc/systemd/network/*.netdev
• nmcli
• /sys/class/net/bonding_masters
• /sys/class/net/bond*/bonding/miimon
• /sys/class/net/bond*/bonding/slaves
• ifenslave
• ip

 

Cluster Overview

 

The cluster comprises four nodes installed with CentOS 7 and housed on a KVM virtual machine system on a Linux Ubuntu host.

 

For this lab I am creating a vlan called vlan-1, for just two machines, ie:

 

ceph-mon
ceph-osd0

 

NOTE: You do NOT need to create a new physical NAT network on KVM, since the VLAN subnet is purely virtual.

 

 

VLAN Tagging

 

Each VLAN is identified by a VID (VLAN Identifier) between 1 and 4094 inclusive. Ports on switches are assigned to a VLAN ID.

 

All ports assigned to a single VLAN are virtually located in their own separate broadcast domain. This reduces network traffic overhead.

 

The VID is stored in a 4-byte header that gets added to the packet, known as the Tag. Hence the name for this procedure is VLAN tagging.

 

 

Configuring VLAN Tagging Using nmcli

 

First ensure the 802.1Q kernel module is loaded. In practice, this module is often automatically loaded if you configure a VLAN subinterface.

 

This is the procedure to manually load it:

 

[root@ceph-mon ~]# modprobe 8021q
[root@ceph-mon ~]#
[root@ceph-mon ~]# lsmod | grep 8021q
8021q 33080 0
garp 14384 1 8021q
mrp 18542 1 8021q
[root@ceph-mon ~]#

 

1. You can use the nmcli connection command to create a VLAN connection.

 

Include the “add type vlan” arguments and any additional information to create a VLAN connection. For example:

 

[root@ceph-mon network-scripts]# nmcli con add type vlan con-name vlan-1 ifname eth0.100 dev eth0 id 100 ip4 192.168.133.40/24
Connection ‘vlan-1’ (25a01a92-740b-481e-8c88-033d6ace0227) successfully added.
[root@ceph-mon network-scripts]#

 

note we create a NEW ifname with eg eth0.100

 

 

nmcli con add type vlan con-name vlan-1 ifname eth0.100 dev eth0 id 100 ip4 192.168.133.40/24

 

 

The example defines the following attributes of the VLAN connection:

 

con-name vlan-1: Specifies the name of the new VLAN connection

 

ifname eth0.100: Specifies the VLAN interface to bind the connection to

 

dev eth0: Specifies the actual physical (parent) device this VLAN is on

 

id 100: Specifies the VLAN ID

 

ip4 192.168.133.1/24: Specifies IPv4 address to assign to the interface

 

 

This command automatically generates the respective network interface config file for the VLAN, so it is preferred to the manual config file method which is documented further below.

 

 

The nmcli con command shows the new VLAN connection.

 

# nmcli connection

 

[root@ceph-mon network-scripts]# nmcli connection
NAME UUID TYPE DEVICE
Wired connection 1 70ed8ab9-f6e1-3180-8d1b-b7c3cb827c8c ethernet eth3
eth0 d1840d20-4b54-49b7-8eb8-305bd11aa5eb ethernet eth0
vlan-1 25a01a92-740b-481e-8c88-033d6ace0227 vlan eth0.100
[root@ceph-mon network-scripts]#

 

this also creates the config file:

 

/etc/sysconfig/network-scripts/ifcfg-vlan-1

 

 

check with:

 

[root@ceph-mon network-scripts]# ls /sys/class/net
bond0 bonding_masters eth0 eth0.100 eth1 eth2 eth3 lo
[root@ceph-mon network-scripts]#

 

and

 

 

[root@ceph-mon network-scripts]# cat ifcfg-vlan-1
VLAN=yes
TYPE=Vlan
PHYSDEV=eth0
VLAN_ID=100
REORDER_HDR=yes
GVRP=no
MVRP=no
HWADDR=
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=192.168.133.40
PREFIX=24
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=vlan-1
UUID=25a01a92-740b-481e-8c88-033d6ace0227
DEVICE=eth0.100
ONBOOT=yes
[root@ceph-mon network-scripts]#

 

 

 

Manual Configuration of Network Interface File for VLAN Tagging

 

 

To manually create the network interface file for the VLAN you need to specify the interface name in the form of parentInterface.vlanID.

 

This associates the VLAN with the appropriate parent network interface. Also set the VLAN=yes directive to define this subinterface as a VLAN.

 

Then restart the network.

 

 

[root@ceph-mon network-scripts]# cat /etc/sysconfig/network-scripts/ifcfg-vlan-1
VLAN=yes
TYPE=Vlan
PHYSDEV=eth0
VLAN_ID=100
REORDER_HDR=yes
GVRP=no
MVRP=no
HWADDR=
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=192.168.133.40
PREFIX=24
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=vlan-1
UUID=25a01a92-740b-481e-8c88-033d6ace0227
DEVICE=eth0.100
ONBOOT=yes
[root@ceph-mon network-scripts]#

 

 

To delete a wifi connection type :

 

nmcli connection delete id <connection name>

 

nmcli connection delete id vlan-1

 

[root@ceph-mon network-scripts]# nmcli connection delete id vlan-1
Connection ‘vlan-1’ (56c10845-07a6-4245-bc95-24c17e991082) successfully deleted.
[root@ceph-mon network-scripts]#

 

 

How to Verify the VLAN Connection

 

 

[root@ceph-mon network-scripts]# ip add show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:93:ca:03 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.40/24 brd 192.168.122.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::6e18:9a8a:652c:1700/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::127d:ea0d:65b7:30e5/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::4ad9:fabb:aad4:9468/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000
link/ether 52:54:00:d7:a5:b0 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000
link/ether 52:54:00:d7:a5:b0 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:22:42:1e brd ff:ff:ff:ff:ff:ff
inet6 fe80::5b5f:1ce3:13:7a74/64 scope link noprefixroute
valid_lft forever preferred_lft forever
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 52:54:00:d7:a5:b0 brd ff:ff:ff:ff:ff:ff
inet 10.0.9.45/24 brd 10.0.9.255 scope global bond0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fed7:a5b0/64 scope link
valid_lft forever preferred_lft forever
7: eth0.100@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 52:54:00:93:ca:03 brd ff:ff:ff:ff:ff:ff
inet 192.168.133.40/24 brd 192.168.133.255 scope global noprefixroute eth0.100
valid_lft forever preferred_lft forever
inet6 fe80::d5c6:9aa5:6996:1635/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@ceph-mon network-scripts]#

 

Note in the above, we can see the newly created vlan interface:

 

7: eth0.100@eth0:

 

[root@ceph-mon network-scripts]# nmcli connection
NAME UUID TYPE DEVICE
Wired connection 1 70ed8ab9-f6e1-3180-8d1b-b7c3cb827c8c ethernet eth3
eth0 d1840d20-4b54-49b7-8eb8-305bd11aa5eb ethernet eth0
vlan-1 25a01a92-740b-481e-8c88-033d6ace0227 vlan eth0.100

 

[root@ceph-mon network-scripts]# nmcli device
DEVICE TYPE STATE CONNECTION
eth0 ethernet connected eth0
eth0.100 vlan connected vlan-1
eth3 ethernet disconnected —
bond0 bond unmanaged —
eth1 ethernet unmanaged —
eth2 ethernet unmanaged —
lo loopback unmanaged —
[root@ceph-mon network-scripts]#

 

 

we can also do:

 

ls /sys/class/net/eth0.100

 

[root@ceph-mon network-scripts]# ls /sys/class/net/eth0.100
addr_assign_type broadcast dev_id duplex ifalias link_mode netdev_group phys_port_name proto_down statistics type
address carrier dev_port flags ifindex lower_eth0 operstate phys_switch_id queues subsystem uevent
addr_len carrier_changes dormant gro_flush_timeout iflink mtu phys_port_id power speed tx_queue_len
[root@ceph-mon network-scripts]#

 

and

 

ls /proc/net/vlan

 

[root@ceph-mon network-scripts]# ls /proc/net/vlan
config eth0.100
[root@ceph-mon network-scripts]#

 

 

 

Configuring Further VLAN Member Nodes

 

 

I then created a VLAN interface on node ceph-osd0 as follows, so that the two nodes (ceph-mon and ceph-osd0) can communicate via the VLAN:

 

[root@ceph-osd0 ~]#
[root@ceph-osd0 ~]# modprobe 8021q
[root@ceph-osd0 ~]# lsmod | grep 8021q
8021q 33080 0
garp 14384 1 8021q
mrp 18542 1 8021q
[root@ceph-osd0 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.122.50 netmask 255.255.255.0 broadcast 192.168.122.255
inet6 fe80::127d:ea0d:65b7:30e5 prefixlen 64 scopeid 0x20<link>
inet6 fe80::6e18:9a8a:652c:1700 prefixlen 64 scopeid 0x20<link>
inet6 fe80::4ad9:fabb:aad4:9468 prefixlen 64 scopeid 0x20<link>
ether 52:54:00:03:66:58 txqueuelen 1000 (Ethernet)
RX packets 40679 bytes 2147951 (2.0 MiB)
RX errors 0 dropped 39457 overruns 0 frame 0
TX packets 817 bytes 54247 (52.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

 

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.9.10 netmask 255.0.0.0 broadcast 10.255.255.255
inet6 fe80::9a5f:c1fc:8228:8d16 prefixlen 64 scopeid 0x20<link>
inet6 fe80::61d0:9d9f:ccc3:9f2e prefixlen 64 scopeid 0x20<link>
inet6 fe80::c466:3844:d978:b3d8 prefixlen 64 scopeid 0x20<link>
ether 52:54:00:a2:a4:1d txqueuelen 1000 (Ethernet)
RX packets 181745 bytes 11234531 (10.7 MiB)
RX errors 0 dropped 39454 overruns 0 frame 0
TX packets 130505 bytes 1040879191 (992.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

 

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 24888 bytes 2206620 (2.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 24888 bytes 2206620 (2.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

 

[root@ceph-osd0 ~]# nmcli con add type vlan con-name vlan-1 ifname eth0.100 dev eth0 id 100 ip4 192.168.133.41/24
Connection ‘vlan-1’ (6c39b373-e1f5-46c2-9137-768f53e5ed22) successfully added.

 

 

[root@ceph-osd0 ~]# nmcli connection
NAME UUID TYPE DEVICE
eth0 d1840d20-4b54-49b7-8eb8-305bd11aa5eb ethernet eth0
eth1 9c92fad9-6ecb-3e6c-eb4d-8a47c6f50c04 ethernet eth1
vlan-1 6c39b373-e1f5-46c2-9137-768f53e5ed22 vlan eth0.100

 

[root@ceph-osd0 ~]# cat /etc/sysconfig/network-scripts/ifcfg-vlan-1
VLAN=yes
TYPE=Vlan
PHYSDEV=eth0
VLAN_ID=100
REORDER_HDR=yes
GVRP=no
MVRP=no
HWADDR=
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=192.168.133.41
PREFIX=24
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=vlan-1
UUID=6c39b373-e1f5-46c2-9137-768f53e5ed22
DEVICE=eth0.100
ONBOOT=yes

 

 

[root@ceph-osd0 ~]# ip add show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:03:66:58 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.50/24 brd 192.168.122.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::6e18:9a8a:652c:1700/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::127d:ea0d:65b7:30e5/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::4ad9:fabb:aad4:9468/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:a2:a4:1d brd ff:ff:ff:ff:ff:ff
inet 10.0.9.10/8 brd 10.255.255.255 scope global noprefixroute eth1
valid_lft forever preferred_lft forever
inet6 fe80::c466:3844:d978:b3d8/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::61d0:9d9f:ccc3:9f2e/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::9a5f:c1fc:8228:8d16/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
4: eth0.100@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 52:54:00:03:66:58 brd ff:ff:ff:ff:ff:ff
inet 192.168.133.41/24 brd 192.168.133.255 scope global noprefixroute eth0.100
valid_lft forever preferred_lft forever
inet6 fe80::497:afcc:dfdd:bafb/64 scope link noprefixroute
valid_lft forever preferred_lft forever

 

[root@ceph-osd0 ~]# ping 192.168.133.40
PING 192.168.133.40 (192.168.133.40) 56(84) bytes of data.
64 bytes from 192.168.133.40: icmp_seq=1 ttl=64 time=1.05 ms
64 bytes from 192.168.133.40: icmp_seq=2 ttl=64 time=0.543 ms
64 bytes from 192.168.133.40: icmp_seq=3 ttl=64 time=0.577 ms
^C
— 192.168.133.40 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 0.543/0.724/1.052/0.232 ms
[root@ceph-osd0 ~]#

 

 

 

I also created a VLAN interface to the vlan-1 VLAN on my laptop (ubuntu):

 

 

Note the interface name is derived from KVM as we are in a KVM virtualized environment. The parent interface is virbr0 and this is the 192.168.122.0 connection to the cluster on KVM from the laptop.

 

 

The VLAN interface “piggybacks” via virbr0 as virbr0.100 with subnet 192.168.133.0

 

(there is no KVM defined subnet for the 192.168.133.0 – it is purely VLAN virtual)

 

root@asus:/home/kevin#
root@asus:/home/kevin# nmcli con add type vlan con-name vlan-1 ifname virbr0.100 dev virbr0 id 100 ip4 192.168.133.1/24
Connection ‘vlan-1’ (e2f09575-95d1-4028-b99b-eb49300bf8b2) successfully added.

 

root@asus:/home/kevin# nmcli con
NAME UUID TYPE DEVICE
vlan-1 e2f09575-95d1-4028-b99b-eb49300bf8b2 vlan virbr0.100

 

root@asus:/etc/netplan# ip add show | grep virbr0
3: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN group default qlen 1000
9: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr0 state UNKNOWN group default qlen 1000
11: vnet2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr0 state UNKNOWN group default qlen 1000
13: vnet4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr0 state UNKNOWN group default qlen 1000
17: vnet6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr0 state UNKNOWN group default qlen 1000
23: virbr0.100@virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 192.168.133.1/24 brd 192.168.133.255 scope global noprefixroute virbr0.100
root@asus:/etc/netplan#

 

root@asus:/etc/netplan# ls /proc/net/vlan
config virbr0.100
root@asus:/etc/netplan#

 

Continue Reading

LPIC3 DIPLOMA Linux Clustering – LAB NOTES: Lesson Network Bonding

By on May 10, 2021 in

 

LAB on Network Bonding

 

These are my notes made during my lab practical as part of my LPIC3 Diploma course in Linux Clustering. They are in “rough format”, presented as they were written.

 

 

LPIC3 Syllabus for Network Bonding

 

364.4 Network High Availability
Weight: 5
Description: Candidates should be able to configure redundant networking connec-
tions and manage VLANs. Furthermore, candidates should have a basic understanding
of BGP.
Key Knowledge Areas:
• Understand and configure bonding network interface
• Network bond modes and algorithms (active-backup, blance-tlb, balance-alb,
802.3ad, balance-rr, balance-xor, broadcast)
• Configure switch configuration for high availability, including RSTP
• Configure VLANs on regular and bonded network interfaces
• Persist bonding and VLAN configuration
• Understand the principle of autonomous systems and BGP to manage external
redundant uplinks
• Awareness of traffic shaping and control capabilities of Linux
 

Partial list of the used files, terms and utilities:
• bonding.ko (including relevant module options)
• /etc/network/interfaces
• /etc/sysconfig/networking-scripts/ifcfg-*
• /etc/systemd/network/*.network
• /etc/systemd/network/*.netdev
• nmcli
• /sys/class/net/bonding_masters
• /sys/class/net/bond*/bonding/miimon
• /sys/class/net/bond*/bonding/slaves
• ifenslave
• ip

 

Cluster Overview

 

The cluster comprises three nodes installed with CentOS 7 and housed on a KVM virtual machine system on a Linux Ubuntu host.

 

Network Card Bonding On CentOS

 

Ethernet network bonding, sometimes known as port trunking or link aggregation is a connection method in which multiple network links are bound together to operate as one single link, effectively combining the bandwidth into a single connection.

 

Linux deploys a special kernel module named bonding to connect multiple network interfaces into a single interface. Most popular Linux distro kernels ship with the bonding driver already available as a module and the ifenslave user level control program installed and ready for use. This can be used to provide redundant links, fault tolerance, load balancing or increased effective bandwidth capacity for a service connection.

 

The two main reasons to use network interface bonding are:

 

1. To provide increased bandwidth
2. To provide redundancy in the event of hardware failure

 

 

Network bonding has different modes. You specify the mode to be used by your bonding interface in /etc/sysconfig/network-scripts/ifcfg-bond0 interfaces file by defining the line bond-mode, for example:

 

bond-mode active-backup

 

or alternatively with:

 

BONDING_OPTS=”mode=1 miimon=100″

 

 

After the channel bonding interface is created, the network interfaces to be bound together must be configured by adding the MASTER and SLAVE directives to their configuration files.

 

 

There are six main modes of bonding:

 

mode=1 (active-backup)
mode=2 (balance-xor)
mode=3 (broadcast)
mode=4 (802.3ad)
mode=5 (balance-tlb)
mode=6 (balance-alb)

 

 

Types of Network Bonding

 

mode=0 (balance-rr)

 

This mode is based on Round-robin policy and it is the default mode. This mode offers fault tolerance and load balancing features. It transmits the packets in Round robin fashion that is from the first available slave through the last.

 

mode-1 (active-backup)

 

This mode is based on Active-backup policy. Only one slave is active in this band, and another one will act only when the other fails. The MAC address of this bond is available only on the network adapter part to avoid confusing the switch. This mode also provides fault tolerance.

 

mode=2 (balance-xor)

 

This mode sets an XOR (exclusive or) mode that is the source MAC address is XOR’d with destination MAC address for providing load balancing and fault tolerance. Each destination MAC address the same slave is selected.

 

mode=3 (broadcast)

 

This method is based on broadcast policy that is it transmitted everything on all slave interfaces. It provides fault tolerance. This can be used only for specific purposes.

 

mode=4 (802.3ad)

 

This mode is known as a Dynamic Link Aggregation mode that has it created aggregation groups having same speed. It requires a switch that supports IEEE 802.3ad dynamic link. The slave selection for outgoing traffic is done based on a transmit hashing method. This may be changed from the XOR method via the xmit_hash_policy option.

 

mode=5 (balance-tlb)

 

This mode is called “adaptive transmit load balancing”. The outgoing traffic is distributed based on the current load on each slave and the incoming traffic is received by the current slave. If the incoming traffic fails, the failed receiving slave is replaced by the MAC address of another slave. This mode does not require any special switch support.

 

mode=6 (balance-alb)

 

This mode is called adaptive load balancing. This mode does not require any special switch support.

 

 

Overview of the bonding mode codes deployed

 

Bonding Mode Bonding Policy Description Fault Tolerance Load balancing
0 Round Robin

Packets are sequentially transmitted/received via each interface in turn. 

 

 No Yes
1 Active Backup

One interface is active while another is asleep. If the active interface fails, another interface takes over. NOTE: Active Backup is only supported in x86 systems. 

 

 Yes No
2 XOR [exclusive OR]

MAC address of the slave interface is matched to the incoming request’s MAC address. Once the connection is made the same interface is then used to transmit/receive for the destination MAC.

 

 Yes Yes
3 Broadcast

All transmissions are sent via all slaves.

 

 Yes No
4 Dynamic Link Aggregation

All interfaces are actively aggregated together as one bonded interface to deliver greater bandwidth and also failover in the case of an interface failure. Dynamic Link Aggregation requires a switch that supports IEEE standard 802.3ad.

 

 Yes Yes
5 Transmit Load Balancing (TLB)

Outgoing network traffic is distributed according to current load on each slave interface. The incoming network traffic is received by the current slave. If the receiving slave interface goes down, then another slave takes over the MAC address from the failed slave interface.

 

 Yes Yes
6 Adaptive Load Balancing (ALB)

In contrast to Dynamic Link Aggregation, Adaptive Load Balancing does not require specific switch configuration. Incoming network traffic is load balanced through ARP negotiation.  NOTE: Adaptive Load Balancing is only supported in x86 systems. 

 

 Yes Yes

 

 

Network Bonding Configuration

 

 

Load the bonding module:

 

modprobe bonding

 

check with:

 

modinfo bonding

 

 

[root@ceph-mon network-scripts]# modinfo bonding
filename: /lib/modules/3.10.0-1160.24.1.el7.x86_64/kernel/drivers/net/bonding/bonding.ko.xz
author: Thomas Davis, tadavis@lbl.gov and many others
description: Ethernet Channel Bonding Driver, v3.7.1
version: 3.7.1
license: GPL
alias: rtnl-link-bond
retpoline: Y
rhelversion: 7.9
srcversion: 3B2F8F8533AEAE2EB01F706
depends:
intree:
.. … .. (long list of output)… ..

 

1) Create the bonding file ( ifcfg-bond0 ) and set the IP address, netmask  and gateway.

 

nano /etc/sysconfig/network-scripts/ifcfg-bond0

 

DEVICE=bond0

IPADDR=10.0.9.45

NETMASK=255.255.255.0

GATEWAY=10.0.9.1

TYPE=Bond

ONBOOT=yes

NM_CONTROLLED=no

BOOTPROTO=static

 

2) Edit the files of eth1 and eth2 and define the master and slave entries: 

 

nano /etc/sysconfig/network-scripts/ifcfg-eth1

 

 

DEVICE=eth1

HWADDR=52:54:00:d7:a5:b0

TYPE=Ethernet

ONBOOT=yes

NM_CONTROLLED=no

MASTER=bond0

SLAVE=yes

 

 

nano /etc/sysconfig/network-scripts/ifcfg-eth2

 

DEVICE=eth2

HWADDR=52:54:00:87:8f:0b

TYPE=Ethernet

ONBOOT=yes

NM_CONTROLLED=no

MASTER=bond0

SLAVE=yes

 

 

3) Create the bonding file bonding.conf

 

nano /etc/modprobe.d/bonding.conf

 

alias bond0 bonding

 

options bond0 mode=1 miimon=100

 

 

4) restart the networking service

 

systemctl restart networking 

 

ifup ifcfg-bond0

 

 

5) To check the bond interface:

 

ifconfig bond0

 

 

6) To verify the status of the bond interface:

 

cat /proc/net/bonding/bond0

 

 

[root@ceph-mon ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

 

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

 

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 52:54:00:d7:a5:b0
Slave queue ID: 0

 

Slave Interface: eth2
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 52:54:00:87:8f:0b
Slave queue ID: 0
[root@ceph-mon ~]#

 

We can see that our bonding mode is set to load balancing (round-robin or rr)

 

and that we are using eth1 and eth2 bonded together as bond0

 

You can also verify with lsmod:

 

[root@ceph-mon ~]# lsmod |grep bond
bonding 152979 0
[root@ceph-mon ~]#

 

 

and with:

 

 

[root@ceph-mon network-scripts]# ip -br address
lo UNKNOWN 127.0.0.1/8 ::1/128
eth0 UP 192.168.122.40/24 fe80::6e18:9a8a:652c:1700/64 fe80::127d:ea0d:65b7:30e5/64 fe80::4ad9:fabb:aad4:9468/64
eth1 UP
eth2 UP
eth3 UP
bond0 UP 10.0.9.45/24 fe80::5054:ff:fed7:a5b0/64
[root@ceph-mon network-scripts]#

 

 

To test fault tolerance:

 

To test the fault tolerance, shut down one interface and check whether you are still able access the server.

 

ifdown eth1

 

or

 

ifdown eth2

 

You should still be able to access the machine via the bond0 interface IP address.

 

 

[root@ceph-mon ~]# ifdown eth1
[root@ceph-mon ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

 

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

 

Slave Interface: eth2
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 52:54:00:87:8f:0b
Slave queue ID: 0
[root@ceph-mon ~]#

 

we can see that eth1 is now no longer part of the bond.

 

bring the interface back up again with

 

ifup eth1

 

 

To Change Bonding Mode

 

 

To change the bonding mode set the BONDING_OPTS value accordingly, eg to set to aggregate bonding (4):

in the interface file:

 

BONDING_OPTS=”mode=4 miimon=100″

 

and then

 

ifdown bond0

 

and

 

ifup bond0

 

verify the change with

 

cat /proc/net/bonding/bond0

 

 

 

Using ifenslave for bonding

 

The tool ifenslave can also be used to configure bonding interfaces. It can be used to attach or detach or change the currently active slave interface from the bonding.

 

to display interface info:

 

[root@ceph-mon network-scripts]# ifenslave -a
The result of SIOCGIFFLAGS on lo is 49.
The result of SIOCGIFADDR is 7f.00.00.01.
The result of SIOCGIFHWADDR is type 772 00:00:00:00:00:00.
The result of SIOCGIFMETRIC is 0
The result of SIOCGIFMTU is 65536
The result of SIOCGIFFLAGS on eth0 is 1043.
The result of SIOCGIFADDR is ffffffc0.ffffffa8.7a.28.
The result of SIOCGIFHWADDR is type 1 52:54:00:93:ca:03.
The result of SIOCGIFMETRIC is 0
The result of SIOCGIFMTU is 1500
The result of SIOCGIFFLAGS on bond0 is 1443.
The result of SIOCGIFADDR is 0a.00.09.2d.
The result of SIOCGIFHWADDR is type 1 52:54:00:d7:a5:b0.
The result of SIOCGIFMETRIC is 0
The result of SIOCGIFMTU is 1500
[root@ceph-mon network-scripts]#

 

 

To create a bond device, follow these three steps :

 

– ensure that the required drivers are properly loaded :
# modprobe bonding ; modprobe <3c59x|eepro100|pcnet32|tulip|…>

 

– assign an IP address to the bond device :

# ifconfig bond0 <addr> netmask <mask> broadcast <bcast>

 

– attach all the interfaces you need to the bond device :
# ifenslave [{-f|–force}] bond0 eth0 [eth1 [eth2]…]

 

If bond0 didn’t have a MAC address, it will take eth0’s. Then, all interfaces attached AFTER this assignment will get the same MAC addr.
(except for ALB/TLB modes)

 

 

-c, –change-active
Change active slave.

 

-d, –detach
Removes slave interfaces from the bonding device.

 

 

To detach an interface from a bond:

 

ifenslave -d <master iface> <slave iface>

 

ifenslave -d bond0 eth2

 

To add an interface to a bond:

 

 

ifenslave <master iface> <slave iface>

 

ifenslave bond0 eth2

 

 

 

To bond 2 interfaces together:

 

ifenslave bond0 eth1 eth2

 

For example, to change the active slave from bond0 to eth2:

 

 

ifenslave -c bond0 eth2

 

 

 

Important: To make changes permanent you must define them in the respective network interfaces config files.

Continue Reading