Cumulus
From Cheatsheet
Jump to navigationJump to search
- Practice: https://www.nvidia.com/en-us/networking/ethernet-switching/cumulus-vx/
- https://docs.nvidia.com/networking-ethernet-software/knowledge-base/Setup-and-Getting-Started/Cumulus-Linux-Command-Reference-Guide/
Checks
Common
# Ping from a specific (VLAN, VRF, ethernet) interface ping -I eth0 10.0.0.2 # Show CLAG status such as names, CLAG IDs, Conflicts and more net show clag status
Interfaces
# Displays significant and relevant information for all 'up' interfaces. net show interface # Show interface information for swp5 net show interface swp5 # List the configuration for an interface. net show configuration interface vlan40 # Show all MAC-adresses living on bridge, interfaces they are found on and their lifetime net show bridge macs # Use /proc/net/ subfolders for VLAN/bond interface information cat /proc/net/vlan/vlan512 cat /proc/net/bonding/server01 # Show interface statistics - drops, transmit, MTU, errors cl-netstat # Show interface statistics - drops, transmit, errors ip -s link # Display statistics for online interfaces netstat -i # Grep the netstat table for a shorter summary netstat -i | grep -e 'swp' -e 'Iface' # Show transceiver information for interface swp4 ethtool swp4 # List all dependencies or bound virtual interfaces of physical and virtual interfaces ifquery --print-dependency=list -a
Routing
# Show all routes net show route # Show the route table ip route show # Show status of all interfaces ip link show # Show all IPs ip addr show
VRFs
# List all existing VRFs vrf list # List all routes for a specific VRF. net show route vrf mgmt # Show links for a specific VRF. ip link show vrf mgmt # Show all IP defined within a specific VRF. ip addr show vrf mgmt # Show routes for a specific VRF. ip route show vrf mgmt # Show the route table for a specific VRF. ip route show table mgmt
Hardware
# Display sensor temperature in Celsius and fans RPM information sensors
Common Configuration
Commands
# Check the configuration for errors and write the new config only if there are no errors. sudo bash -c "ifreload -s -a && ifreload -a"
Syslog
# /etc/rsyslog.d/70-local-to-rsyslog-server.conf # Define a template and specify a hostname to send as: template(name="SendHostname" type="string" string="%timestamp% myhost.mydomain.nl %syslogtag% %msg%\n" ) # Send all logs of severity warning, to target syslog server and port, via interface vlan33 *.warning action(type="omfwd" Target="10.0.33.10" Template="SendHostname" Device="vlan33" Port="514" Protocol="udp")
Network
Interfaces
File-location: /etc/network/interfaces
Management
auto eth0
iface eth0
address 192.168.0.1/24
vrf mgmt
auto mgmt
iface mgmt
address 127.0.0.1/8
vrf-table auto
VLAN MGMT Interface
auto eth0
iface eth0
auto mgmt
iface mgmt
address 127.0.0.1/8
vrf-table auto
auto vlan66
iface vlan66
vlan-id 66
vlan-raw-device eth0
address 10.66.0.101/24
vrf mgmt
Regular
auto swp4 iface swp4 auto swp5 iface swp5
Inactive
auto swp8
iface swp8
link-down yes
IP
auto swp2
iface swp2 inet static
address 172.16.25.1/24
VLAN
Via physical interfaces
auto swp1
iface swp1
auto swp2
iface swp2
auto bridge
iface bridge
bridge-vlan-aware yes
bridge-ports swp1 swp2
bridge-vids 50 60-70
bridge-pvid 1
bridge-stp on
Via child interfaces
auto swp1.100
iface swp1.100
auto swp2.100
iface swp2.100
auto br-100
iface br-100
bridge-ports swp1.100 swp2.100
bridge-stp on
VLAN IP Interfaces
auto swp28
iface swp28
auto swp29
iface swp29
auto vlan88
iface vlan88
vlan-id 88
vlan-raw-device bridge
address 192.168.88.200/24
auto bridge
iface bridge
bridge-ports swp28 swp29
bridge-vids 88
bridge-vlan-aware yes
LACP
1 switch to 2 interfaces on a single server:
auto swp1
iface swp1
alias downlink to server01 port eth0
auto swp2
iface swp2
alias downlink to server01 port eth1
auto bond1
iface bond1
bond-mode 802.3ad
bond-slaves swp1 swp2
bridge-vids 99
auto bridge
iface bridge
bridge-ports bond1
bridge-vids 99
bridge-vlan-aware yes
CLAG/VPC link
- https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-43/Layer-2/Multi-Chassis-Link-Aggregation-MLAG/#
- https://www.redpill-linpro.com/techblog/2018/02/26/layer3-cumulus-mlag.html
Switch #1
auto swp1
iface swp1
alias peerlink to switch-2 port swp1
auto swp2
iface swp2
alias peerlink to switch-2 port swp2
auto swp20
iface swp20
alias downlink to server01 port eth0
# Add any required physical or VLAN interfaces to the bond and define a unique clag-id
auto myserver01
iface myserver01
alias server01-MLAG
bond-slaves swp20
bridge-vids 55 66 77
bridge-allow-untagged no
clag-id 1
# Define peerlink and myserver01 on the bridge interface
# On Mellanox switches, you must add all VLANs configured on the MLAG bond to the bridge so that traffic to the downstream device connected in MLAG is redirected successfully over the peerlink in case of an MLAG bond failure.
auto bridge
iface bridge
bridge-ports myserver01 peerlink
bridge-vids 55 66 77
bridge-vlan-aware yes
# Interfaces connecting to your peer switch
auto peerlink
iface peerlink
bond-slaves swp1 swp2
# clagd-backup-ip is purely for connectivity between switches
# For clagd-sys-mac pick a unique MAC between 44:38:39:ff:00:00 and 44:38:39:ff:ff:ff per MLAG
# linklocal defines an independent unroutable IP
# clagd-priority for defining the primary switch. Range between 0 and 65535, lower value is 'more' primary
auto peerlink.4094
iface peerlink.4094
clagd-peer-ip linklocal
clagd-backup-ip 192.168.255.1
clagd-sys-mac 44:38:39:FF:00:01
clagd-priority 1000
Switch #2
auto swp1
iface swp1
alias peerlink to switch-1 port swp1
auto swp2
iface swp2
alias peerlink to switch-1 port swp2
auto swp20
iface swp20
alias downlink to server01 port eth1
# Add any required physical or VLAN interfaces to the bond and define a unique clag-id
auto myserver01
iface myserver01
alias server01-MLAG
bond-slaves swp20
bridge-vids 55 66 77
bridge-allow-untagged no
clag-id 1
# Define peerlink and bondsite1 on a bridge interface
auto bridge
iface bridge
bridge-ports myserver01 peerlink
bridge-vlan-aware yes
# Interfaces connecting to your peer switch
auto peerlink
iface peerlink
bond-slaves swp1 swp2
# clagd-backup-ip is purely for connectivity between switches
# For clagd-sys-mac pick a unique MAC between 44:38:39:ff:00:00 and 44:38:39:ff:ff:ff per MLAG
# linklocal defines an independent unroutable IP
# clagd-priority for defining the primary switch. Range between 0 and 65535, lower value is 'more' primary
auto peerlink.4094
iface peerlink.4094
clagd-peer-ip linklocal
clagd-backup-ip 192.168.255.2
clagd-sys-mac 44:38:39:FF:00:01
clagd-priority 2000