LPIC3 DIPLOMA Linux Clustering – LAB NOTES: Lesson RSTP

You are here:
< All Topics

LAB on RSTP

 

These are my notes made during my lab practical as part of my LPIC3 Diploma course in Linux Clustering. They are in “rough format”, presented as they were written.

 

 

LPIC3 Syllabus for RSTP

 

364.4 Network High Availability
Weight: 5
Description: Candidates should be able to configure redundant networking connections and manage VLANs.

Furthermore, candidates should have a basic understanding of BGP.

Key Knowledge Areas:
• Understand and configure bonding network interface
• Network bond modes and algorithms (active-backup, blance-tlb, balance-alb,
802.3ad, balance-rr, balance-xor, broadcast)
• Configure switch configuration for high availability, including RSTP
• Configure VLANs on regular and bonded network interfaces
• Persist bonding and VLAN configuration
• Understand the principle of autonomous systems and BGP to manage external
redundant uplinks
• Awareness of traffic shaping and control capabilities of Linux
 

Partial list of the used files, terms and utilities:
• bonding.ko (including relevant module options)
• /etc/network/interfaces
• /etc/sysconfig/networking-scripts/ifcfg-*
• /etc/systemd/network/*.network
• /etc/systemd/network/*.netdev
• nmcli
• /sys/class/net/bonding_masters
• /sys/class/net/bond*/bonding/miimon
• /sys/class/net/bond*/bonding/slaves
• ifenslave
• ip

 

Cluster Overview

 

The cluster comprises four nodes installed with CentOS 7 and housed on a KVM virtual machine system on a Linux Ubuntu host.

 

For this lab I am creating a vlan called vlan-1, for just two machines, ie:

 

ceph-mon
ceph-osd0

 

An Overview of Rapid Spanning Tree Protocol (RSTP) / Spanning Tree Protocol (STP)

 

 

RSTP is a further development of STP or Spanning Tree Protocol, which was the original switching protocol for managing switching in Ethernet networks. RSTP has a number of modifications to STP, but the principles remain largely the same. So to understand RSTP we need first understand the basics of STP.

 

STP is used to prevent network traffic loops which can occur as networks grow and use multiple switches

 

Broadcast messages are being sent all the time on an Ethernet network and this means “Broadcast Storms” can easily occur which generate substantial network traffic overhead and result in slow network connectivity

 

Loops can also result in unstable MAC address tables since they are being constantly updated and this can result in duplicate Ethernet frames being sent to the same host/s.

 

The original STP is also known as IEEE 802.1d

 

 

To avoid this happening, STP-based switches run an “Election Process” to elect a master switch or “Root Bridge” (RB) which plays the key role in the switching network.

 

How this happens in overview:

 

The switch with the lowest BID is elected RB

 

All Root Bridge interfaces are placed into a Forwarding state

 

The Non-Root switches each elect their own Root Port, based on:
– Lowest cost
– Lowest neighbour BID
– Lowest neighbour port priority
– Lowest neighbour port number

 

and then elect a Designated Port, based on:
– Lowest root cost
– Lowest BID
– Lowest neighbour port priority
– Lowest neighbour port number

 

And finally, all the other ports are placed into a Blocking State

 

Here are some key definitions used in STP/RSTP:

 

STP/RSTP Roles and States

 

Roles

 

– Root Ports: this is the best port on each switch to reach the Root Bridge

– Designated Port: this is the port that offers the best route to the Root Bridge

– Non-Designated Ports: all the other ports which are in a blocking state

 

States

– Disabled: The port that is currently shutdown
– Blocking: A port that is currently blocking traffic. This port must first move to Listening state before it can move to Forwarding state
– Listening: A port that is not forwarding traffic and is not learning MAC addresses. This is a transitional State used while switching from one role to another. It is held in this state for the Forward Delay timer (which by default is 15 secs)
– Learning: A port which is not forwarding traffic but IS learning MAC addresses. This is a transitional State for when changing from one role to another. Again, it is held in this state for the Forward Delay timer of 15 sec default. From here, the port can then move on to the Forwarding state.

– Forwarding : A port that is sending and receiving traffic as per normal. From here the port can move directly to Blocking state.

 

RSTP/STP Root Bridge Election

 

Each switch generates “BPDUs” or Bridge Protocol Data Units, which are messages sent to all switches in the network. They are used to select a single switch to act as Root Bridge as the master or root of the spanning tree.  BPDUs help calculate the shortest path from itself to the Root Bridge, designated one of the switches as the closest to the Root for each LAN segment.

 

To do this, BPDUs help switches use the following:

 

– Root Cost: the cost of the root bridge
– BID (Bridge ID): this is the switch with the lowest overall BID that will become the root bridge. BIDs are in the format: 32769aaaa:aaaa:aaaa and are comprised of:
– STP priority: This has a default value of 32768 + the VLAN ID number. Eg for VLAN 1, the STP priority would be 32769. – MAC Address

 

Each switch shares its BPDUs with the others. They use these to elect the Root Bridge. 

 

Then: all ports on the root bridge enter a Forwarding State, and each non-root switch then chooses the best path to the root bridge.

 

This is known as the Root Port and is based on Port Cost. The Port Cost in turn is based on port speed: the higher the speed, the lower the cost. – Each outgoing port to the root is summed together, although it can also be set manually. If there is a tie, then they will examine the lowest neighbor BID.

 

If that also results in a tie, then they look at lowest neighbor port priority to decide.

 

To select a Designated Port, the lowest root cost to the bridge will decide. If that is a tie, then lowest BID is used. If that is also a tie ,then lowest neighbor port priority. And if that also ties, then it will be decided according to lowest neighbor port number.

 

Each port that is not a root port or designated port is put into a Blocking State

 

 

Timers are used for STP:

 

– Hello: occurs 2 seconds and informs all the switches that everything is still active and running
– MaxAge: This is 10 x Hello timer by default (ie 20 seconds) and is the time the switch will wait before it decides there is a problem
– Forward Delay: This is 15 seconds and is the time between the Listening and Learning state

 

 

One problem with STP is that of Convergence. This refers to the time it takes for the STP switching network to update itself to changes in the network eg new switches, switches going offline for whatever reason, changed routes, etc, and to become stable again.

 

To resolve this problem, RSTP was developed to provide for much faster convergence.

 

RSTP was introduced by IEEE in 2001 as IEEE standard 802.1w. RSTP is backwards-compatible with standard STP.

 

While STP can take 30 to 50 seconds to respond to network topology changes, RSTP can respond to changes within 3 × Hello times (default, meaning 3 times 2 seconds) or even within a few milliseconds in the case of a physical link failure.

 

RSTP adds new bridge port roles in order to speed convergence following a link failure. The number of states a port can be in has been reduced to three instead of STP’s original five.

 

RSTP bridge port roles are as follows:

 

Root – A forwarding port that is the best port from non-root bridge to root bridge
Designated – A forwarding port for every LAN segment
Alternate – An alternate path to the root bridge. This path is different from using the root port
Backup – A backup/redundant path to a segment where another bridge port already connects
Disabled – Not strictly part of STP, a network administrator can manually disable a port

 

 

RSTP switch port states are as follows:

 

Discarding – No user data is sent over the port
Learning – The port is not forwarding frames yet, but is populating its MAC-address-table
Forwarding – The port is fully operational

 

 

Note that Cisco developed, implemented and published the Per-VLAN Spanning Tree (PVST) as its own Cisco-proprietary protocol which uses its own proprietary Inter-Switch Link (ISL) system for VLAN encapsulation, as well as PVST+ which uses 802.1Q VLAN encapsulation.

 

Both of these standards implement a separate spanning tree for each VLAN. Cisco switches commonly implement PVST+ and can only implement Spanning Trees for VLANs if the other switches in the LAN also use the same VLAN STP protocol.

 

Cisco also published its own proprietary version of Rapid Spanning Tree Protocol. This creates a spanning tree for each VLAN, like PVST. The Cisco term for this is Rapid Per-VLAN Spanning Tree (RPVST).

 

Commands and Practical Examples of RSTP/STP Configuration

 
 
To list current network bridges on a machine:
 
 
These are all virtual bridges (virbrN) or virtual interfaces (vnetN) created in either KVM or for vlan tagging in the case of vlan-1
 
 
virbr2 is the bridge to the ceph cluster (the 10.0.9.0 subnet) and this is the one we will be working with.
 
 
root@asus:~# bridge link show
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 master virbr0 state disabled priority 32 cost 100 
6: virbr1-nic: <BROADCAST,MULTICAST> mtu 1500 master virbr1 state disabled priority 32 cost 100 
8: virbr2-nic: <BROADCAST,MULTICAST> mtu 1500 master virbr2 state disabled priority 32 cost 100 
9: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master virbr0 state forwarding priority 32 cost 100 
10: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master virbr2 state forwarding priority 32 cost 100 
11: vnet2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master virbr0 state forwarding priority 32 cost 100 
12: vnet3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master virbr2 state forwarding priority 32 cost 100 
13: vnet4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master virbr0 state forwarding priority 32 cost 100 
14: vnet5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master virbr2 state forwarding priority 32 cost 100 
17: vnet6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master virbr0 state forwarding priority 32 cost 100 
18: vnet7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master virbr2 state forwarding priority 32 cost 100 
19: vnet8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master virbr2 state forwarding priority 32 cost 100 
20: vnet9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master virbr2 state forwarding priority 32 cost 100 
root@asus:~#
 
 
root@asus:/# nmcli connection show –active
NAME           UUID                                  TYPE    DEVICE     
vlan-1         e2f09575-95d1-4028-b99b-eb49300bf8b2  vlan    virbr0.100 
virbr0         5250185b-2f90-45ed-bcae-58fce5c411eb  bridge  virbr0     
virbr1         bc49c417-7588-4373-90db-844ff89ed2c5  bridge  virbr1     
virbr2         a0218995-b00a-43e4-bec0-575d0e58c18e  bridge  virbr2     
vnet0          49e9c469-fac4-43bb-86c7-3f2260b8e8b8  tun     vnet0      
vnet1          81435954-b950-4828-8c2e-0204787b57c3  tun     vnet1      
vnet2          03d08db2-54cd-46da-b401-361ee387a675  tun     vnet2      
vnet3          78505fa5-a3ba-4411-86b7-cfeb07aaed39  tun     vnet3      
vnet4          55724fa8-17ef-4c9c-9734-c004146b8980  tun     vnet4      
vnet5          dbfcba77-4a64-419e-8f1d-da08b68ca597  tun     vnet5      
vnet6          5229e91a-640e-478c-807f-da256b7f8322  tun     vnet6      
vnet7          08baf3e0-8e86-4935-a96f-d6115052b695  tun     vnet7      
vnet8          7139dcdd-cc3a-4e78-b52b-785f9c2f1fa0  tun     vnet8      
vnet9          ecebba71-93a1-461d-9ad3-61036b89541c  tun     vnet9      
root@asus:/home/kevin# 
 
 
root@asus:/# ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 144721  bytes 14116503 (14.1 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 144721  bytes 14116503 (14.1 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
 
virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:12:7e:76  txqueuelen 1000  (Ethernet)
        RX packets 5093  bytes 283079 (283.0 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4879  bytes 372044 (372.0 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
 
virbr1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 10.0.8.1  netmask 255.255.255.0  broadcast 10.0.8.255
        ether 52:54:00:07:51:14  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
 
virbr2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.9.1  netmask 255.255.255.0  broadcast 10.0.9.255
        ether 52:54:00:af:7d:f9  txqueuelen 1000  (Ethernet)
        RX packets 141583  bytes 5680986 (5.6 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 9492  bytes 969590 (969.5 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
 
virbr0.100: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.133.1  netmask 255.255.255.0  broadcast 192.168.133.255
        inet6 fe80::f251:4688:9f4d:42aa  prefixlen 64  scopeid 0x20<link>
        ether 52:54:00:12:7e:76  txqueuelen 1000  (Ethernet)
        RX packets 6  bytes 336 (336.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 97  bytes 16520 (16.5 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
 
To disable or enable STP for a network bridge
 
 
To disable STP:
 
 
root@asus:~# nmcli con modify virbr2 bridge.stp no
root@asus:~# nmcli -f bridge con show virbr2
bridge.mac-address:                     —
bridge.stp:                             no
bridge.priority:                        32768
bridge.forward-delay:                   2
bridge.hello-time:                      2
bridge.max-age:                         20
bridge.ageing-time:                     300
bridge.group-forward-mask:              0
bridge.multicast-snooping:              yes
bridge.multicast-startup-query-interval:3124
bridge.vlan-filtering:                  no
bridge.vlan-default-pvid:               1
bridge.vlans:                           —
root@asus:~# 
 
 
To enable STP:
 
 
root@asus:~# nmcli con modify virbr2 bridge.stp yes
root@asus:~# nmcli -f bridge con show virbr2
bridge.mac-address:                     —
bridge.stp:                             yes
bridge.priority:                        32768
bridge.forward-delay:                   2
bridge.hello-time:                      2
bridge.max-age:                         20
bridge.ageing-time:                     300
bridge.group-forward-mask:              0
bridge.multicast-snooping:              yes
bridge.multicast-startup-query-interval:3124
bridge.vlan-filtering:                  no
bridge.vlan-default-pvid:               1
bridge.vlans:                           —
root@asus:~# 
 
 
 
Verify with: 
 
 
nmcli -f bridge con show virbr2    (see above)
 
 
 
brctl stands for Bridge Control. Use the brctl tool to display further information (this command is part of the bridge-utils package, which may need to first be installed. On Ubuntu, with:
 
 
apt install bridge-utils
 
 
 
root@asus:~# brctl show
bridge name bridge id STP enabled interfaces
virbr0 8000.525400127e76 yes virbr0-nic
vnet0
vnet2
vnet4
vnet6
virbr1 8000.525400075114 yes virbr1-nic
virbr2 8000.525400af7df9 yes virbr2-nic
vnet1
vnet3
vnet5
vnet7
vnet8
vnet9
root@asus:~# 
 
 
root@asus:~# brctl showstp virbr2
virbr2
 bridge id 8000.525400af7df9
 designated root 8000.525400af7df9
 root port    0 path cost    0
 max age   20.00 bridge max age   20.00
 hello time    2.00 bridge hello time    2.00
 forward delay    2.00 bridge forward delay    2.00
 ageing time 300.00
 hello timer    1.42 tcn timer    0.00
 topology change timer    0.00 gc timer 106.37
 flags
 
 
 
virbr2-nic (1)
 port id 8001 state        disabled
 designated root 8000.525400af7df9 path cost 100
 designated bridge 8000.525400af7df9 message age timer    0.00
 designated port 8001 forward delay timer    0.00
 designated cost    0 hold timer    0.00
 flags
 
 
vnet1 (2)
 port id 8002 state      forwarding
 designated root 8000.525400af7df9 path cost 100
 designated bridge 8000.525400af7df9 message age timer    0.00
 designated port 8002 forward delay timer    0.00
 designated cost    0 hold timer    0.42
 flags
 
 
vnet3 (3)
 port id 8003 state      forwarding
 designated root 8000.525400af7df9 path cost 100
 designated bridge 8000.525400af7df9 message age timer    0.00
 designated port 8003 forward delay timer    0.00
 designated cost    0 hold timer    0.42
 flags
 
 
vnet5 (4)
 port id 8004 state      forwarding
 designated root 8000.525400af7df9 path cost 100
 designated bridge 8000.525400af7df9 message age timer    0.00
 designated port 8004 forward delay timer    0.00
 designated cost    0 hold timer    0.42
 flags
 
 
vnet7 (5)
 port id 8005 state      forwarding
 designated root 8000.525400af7df9 path cost 100
 designated bridge 8000.525400af7df9 message age timer    0.00
 designated port 8005 forward delay timer    0.00
 designated cost    0 hold timer    0.42
 flags
 
 
vnet8 (6)
 port id 8006 state      forwarding
 designated root 8000.525400af7df9 path cost 100
 designated bridge 8000.525400af7df9 message age timer    0.00
 designated port 8006 forward delay timer    0.00
 designated cost    0 hold timer    0.42
 flags
 
 
vnet9 (7)
 port id 8007 state      forwarding
 designated root 8000.525400af7df9 path cost 100
 designated bridge 8000.525400af7df9 message age timer    0.00
 designated port 8007 forward delay timer    0.00
 designated cost    0 hold timer    0.42
 flags
 
root@asus:~# 
 
 
 
 
 
Using brctl showmacs, you can display all the currently learned MAC addresses of a bridge. This data continually changes according to the current status of what is connected to the bridge.
 
 
root@asus:~# brctl showmacs virbr2
port no mac addr is local? ageing timer
  7 52:54:00:22:42:1e no 247.40
  4 52:54:00:a2:a4:1d no    0.55
  3 fe:54:00:05:9d:32 yes    0.00
  3 fe:54:00:05:9d:32 yes    0.00
  7 fe:54:00:22:42:1e yes    0.00
  7 fe:54:00:22:42:1e yes    0.00
  6 fe:54:00:87:8f:0b yes    0.00
  6 fe:54:00:87:8f:0b yes    0.00
  4 fe:54:00:a2:a4:1d yes    0.00
  4 fe:54:00:a2:a4:1d yes    0.00
  2 fe:54:00:b3:3a:2a yes    0.00
  2 fe:54:00:b3:3a:2a yes    0.00
  5 fe:54:00:d7:a5:b0 yes    0.00
  5 fe:54:00:d7:a5:b0 yes    0.00
root@asus:~# 
 
 
 
With  brctl addif  you can add an interface to an existing ethernet bridge.
 
 
eg
 
brctl addif dev eth0
 
 
This will cause eth0 to act as a port of of the bridge “dev”. Thus all traffic that arrives on eth0 will be processed as if it is really arriving at the bridge “dev”.  Also, when traffic is outgoing from “dev” bridge, it will use eth0.
 
 
To delete a network bridge use:
 
 brctl delbr dev

 

Table of Contents