Link Aggregation
This chapter describes how to configure link aggregation groups between switches, between switches and tools, or between switches and taps.
Configuring Link Aggregation
DMF provides a configurable method of hashing for load distribution among the members of a LAG. The enhanced hashing algorithm automatically assigns the best hashing type for the switch and traffic. This setting also lets you manually select the packet types and fields used for load distribution among the members of a port-channel interface. For the supported switch platforms, enhanced mode and symmetric hashing are enabled by default. With symmetric hashing, bidirectional traffic between two hosts going out on a port channel is distributed on the same member port.
- IPv4
- IPv6
- MPLS (disabled by default)
- L2GRE packet
- hash l2 dst-mac eth-type src-mac vlan-id
- hash ipv4 dst-ip src-ip
- hash ipv6 dst-ip src-ip
- hash l2gre inner-l3 dst-ip src-ip
- hash symmetric
Using the GUI to Configure Link Aggregation Groups
Using the CLI to Configure Link Aggregation Groups
Configuring Hashing Fields
To configure the hashing fields manually via the CLI, use the lag-enhanced-hash command to enter config-switch-hash mode as in the following example:
controller1(config)# switch DMF-FILTER-SWITCH-1
controller1(config-switch)# lag-enhanced-hash
controller1(config-switch-hash)#
- To hash on GTP fields, pick one of the following options:
controller1(config-switch-hash)# hash gtp header-first-byte Configure fields to identify GTP traffic port-match Configure UDP tunnel port match entry
- To hash on IPv4 fields, pick one of the following options:
controller1(config-switch-hash)# hash ipv4 <cr> dst-ip Destination IPv4 address (optional) l4-dst-port TCP/UDP destination port (optional) l4-src-port TCP/UDP source port (optional) protocol IP protocol (optional) src-ip Source IPv4 address (optional) vlan-id Vlan Id (optional)
- To hash on IPv6 fields, pick one of the following options
controller1(config-switch-hash)# hash ipv6 <cr> dst-ip Collapsed destination IPv6 address (optional) l4-dst-port TCP/UDP destination port (optional) l4-src-port TCP/UDP source port (optional) nxt-hdr Next Header (optional) src-ip Collapsed source IPv6 address (optional) vlan-id Vlan Id (optional)
- To hash on Layer-2 fields, pick one of the following options:
controller1(config-switch-hash)# hash l2 dst-mac Destination xMAC address eth-type Ethernet Type src-mac Source MAC address vlan-id Vlan Id
- To hash on L2GRE fields, pick one of the following options:
controller1(config-switch-hash)# hash l2gre inner-l2 Use inner L2 fields for hash computation (optional) inner-l3 Use inner L3 fields for hash computation (optional)
- To hash on MPLS labels, pick one of the following options:
controller1(config-switch-hash)# hash mpls <cr> label-1 Lower 16 bits of MPLS label 1 (optional) label-2 Lower 16 bits of MPLS label 2 (optional) label-3 Lower 16 bits of MPLS label 3 (optional) label-hi-bits Higher 4 bits of MPLS Labels 1,2 and 3 (optional)
- To manually configure the hash seeds:
controller1(config-switch)# hash seeds <First hash seed> Configure seed1 for hash computation controller1(config-switch-hash)# hash seeds 3809 <cr> <Second hash seed> Configure seed2 for hash computation (optional) controller1(config-switch-hash)# hash seeds 3809 90901 <cr>
- To enable/disable symmetric hashing
controller1(config-switch-hash)# hash symmetric <cr> disable Disable symmetric hashing enable Enable symmetric hashing
L2 GRE Key Hashing
Previously, L2 GRE payload-based hashing (InnerL2 or InnerL3) applied only to L2 GRE packets terminated at DMF delivery or filter switches. If a user wanted to hash L2 GRE packets transiting a DMF core switch, the L2 GRE payload-based hashing across port-channel interfaces would not have been functional as the L2 GRE tunnel was not terminating on the core DMF switch.
With the L2 GRE Key-based hashing feature, users can now hash L2 GRE packets based on the L2 GRE Key on core DMF switches.CLI Configuration
L2 GRE Key-based hashing is supported only for the IPv4-based packets with L2 GRE payload. This feature does NOT support the IPv6 packets with L2 GRE payloads.Enable the L2 GRE Key hashing by setting the l2-gre-key
parameter as shown in the following example.
Controller-Active# show running-config switch DMF-SWITCH-1
! switch
switch DMF-SWITCH-1
mac c0:d6:82:17:fd:5a
!
lag-enhanced-hash
hash ipv4 l2-gre-key
hash symmetric disable
GUI Configuration
CLI Commands
Use the following CLI commands to verify settings and to troubleshoot any issues that may arise.
# show lag-enhanced-hash
While logged into a switch use the following commands to troubleshoot this feature.
root@DMF-SWITCH-1:~# ofad-ctl gt PORT_CHANNEL_ENHANCED_HASH_FIELD
Hash Field Configs:
-------------------
Symmetric Hashing:
Disabled
L2GRE Key Hashing:
Enabled
L2 Fields:
IPv4 Fields:
DSTL4 SRCL4
IPv6 Fields:
MPLS Fields:
L2GRE L2 Fields:
L2GRE L3 Fields:
VXLAN L2 Fields:
VXLAN L3 Fields:
root@DMF-SWITCH-1:~# ofad-ctl bshell getreg RTAG7_HASH_CONTROL_L2GRE_MASK_A
RTAG7_HASH_CONTROL_L2GRE_MASK_A.ipipe0[1][0x6a001900]=0xffffffff
: <L2GRE_TUNNEL_GRE_KEY_MASK_A=0xffffffff>
root@DMF-SWITCH-1:~# ofad-ctl bshell getreg RTAG7_HASH_CONTROL_L2GRE_MASK_B
RTAG7_HASH_CONTROL_L2GRE_MASK_B.ipipe0[1][0x6a001a00]=0xffffffff
: <L2GRE_TUNNEL_GRE_KEY_MASK_B=0xffffffff>
root@s5248f-1:~# ofad-ctl bshell getreg RTAG7_L2GRE_PAYLOAD_L2_HASH_FIELD_BMAP
RTAG7_L2GRE_PAYLOAD_L2_HASH_FIELD_BMAP.ipipe0[1][0x6a001b00]=0: <
L2GRE_PAYLOAD_L2_BITMAP_B=0,L2GRE_PAYLOAD_L2_BITMAP_A=0>
root@s5248f-1:~# ofad-ctl bshell getreg RTAG7_L2GRE_PAYLOAD_L3_HASH_FIELD_BMAP
RTAG7_L2GRE_PAYLOAD_L3_HASH_FIELD_BMAP.ipipe0[1][0x6a001c00]=0: <
L2GRE_PAYLOAD_L3_BITMAP_B=0,L2GRE_PAYLOAD_L3_BITMAP_A=0>
root@DMF-SWITCH-1:~#
VxLAN Hashing
Symmetric hashing works with VxLAN packet Inner L3 Source IP/Destination IP, Inner L4 Source Port/Destination Port, and Outer L3 Source IP/Destination IP.
CLI Configuration
VxLAN hashing includes hashing on L2 and L3 and the setting of at least one parameter enabled under the switch construct on Controller CLI:# lag-enhanced-hash
hash vxlan inner-l2 dst-mac
hash vxlan inner-l3 dst-ip
UI Configuration
CLI Commands
Use the following CLI commands to verify settings and to troubleshoot any issues that may arise.
# show lag-enhanced-hash
Use the following commands to troubleshoot this feature. For example, when the hashing happens on VxLAN payload inner L3 Src IP.
root@DMF-SWITCH-1:~# root@mrv1:~# ofad-ctl gt PORT_CHANNEL_ENHANCED_HASH_FIELD
Hash Field Configs:
-------------------
Symmetric Hashing:
Disabled
L2 Fields:
IPv4 Fields:
IPv6 Fields:
MPLS Fields:
L2GRE L2 Fields:
L2GRE L3 Fields:
VXLAN L2 Fields:
VXLAN L3 Fields:
IP4SRC_LO IP4SRC_HI
root@mrv1:~# ofad-ctl bshell getreg RTAG7_HASH_CONTROL_4
RTAG7_HASH_CONTROL_4.ipipe0[1][0x6a000700]=3:
<VXLAN_PAYLOAD_HASH_SELECT_B=1,VXLAN_PAYLOAD_HASH_SELECT_A=1,DISABLE_HASH_VXLAN_B=0,DISABLE_HASH_VXLAN_A=0>
root@mrv1:~# ofad-ctl bshell getreg RTAG7_VXLAN_PAYLOAD_L2_HASH_FIELD_BMAP
RTAG7_VXLAN_PAYLOAD_L2_HASH_FIELD_BMAP.ipipe0[1][0x6a001d00]=0: <
VXLAN_PAYLOAD_L2_BITMAP_B=0,VXLAN_PAYLOAD_L2_BITMAP_A=0>
root@mrv1:~# ofad-ctl bshell getreg RTAG7_VXLAN_PAYLOAD_L3_HASH_FIELD_BMAP
RTAG7_VXLAN_PAYLOAD_L3_HASH_FIELD_BMAP.ipipe0[1][0x6a001e00]=0x1800c00
: <VXLAN_PAYLOAD_L3_BITMAP_B=0xc00,VXLAN_PAYLOAD_L3_BITMAP_A=0xc00>
MLAG RTAG7 Hash Computation for Unbalanced Load Balancing
In DANZ Monitoring Fabric (DMF), all SWL OS switches with the same underlying ASIC have the same RTAG7 hash parameters configured by default. After configuring MLAG to load balance traffic from filter to delivery switches, and when the number of interfaces in the MLAG and LAG in the delivery switch is the same, the traffic will not be load balanced in the delivery switch because of hash polarization.
Use the feature to avoid LAG hash polarization by choosing different hash algorithms in separate switches. The other option is using a different packet field set in each switch.
The feature supports using different hash algorithms by choosing different hash seed values.
CLI Configuration
(config)# switch S4048T
(config-switch)# lag-enhanced-hash
(config-switch)# hash symmetric enable
(config-switch-hash)# hash seeds 1234 3456
(config-switch-hash)# hash l2 dst-mac
(config-switch-hash)# hash l2 src-mac
(config-switch-hash)# hash l2 eth-type
(config-switch-hash)# hash l2 vlan-id
show
command to view the configured hash seed.
R450-C1# show lag-enhanced-hash switch S4048T
~~~~~~~~~~~~~~~~~~~~~~~~~ L2 Enhanced Hash~~~~~~~~~~~~~~~~~~~~~~~~~
# Switch DPID Dst mac Eth type Src mac Vlan id Seed1 Seed2 Symmetric
-|-----------|-------|--------|-------|-------|-----|-----|---------|
1 S4048TTrueTrue TrueTrue12343456True
GUI Configuration
From the home page, select the switches using
.A list of available switches displays.
Choose the required switch for configuring the hash seeds. Hash seeds are integers which are used in generating random numbers in the RTAG7 hash algorithm.
Select Actions from the menu.
On the Configure Switch page, select LAG Enhanced Hash.
- Options
- Symmetric
- L2 Fields
- Src. MAC
- Dst. MAC
- Ether-Type
- VLAN ID
Optional: Use the CLI, and the following show
command to view the configured hash seed.
R450-C1# show lag-enhanced-hash
~~~~~~~~~~~~~~~~~~~~~~~~~~~ L2 Enhanced Hash~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Switch DPID Dst mac Eth type Src mac Vlan id Seed1 Seed2 Symmetric
-|---------------|-------|--------|-------|-------|-----|-----|---------|
1 A-7050SX3-48YC8 TrueTrue TrueTrue12343456True
Troubleshooting a Configured Hash Seed
- Log in to the switch using the
connect switch switch_name
command. - Use the
ofad-ctl gt PORT_CHANNEL_ENHANCED_HASH_SEED
command to print the seed values and the hash algorithm. - Ensure the hash algorithms differ in the switches where hash polarization occurs.
- Configure the hash seeds such that the hash algorithms are different.
R450-C1(config)# connect switch A-7050SX3-48YC8
Switch Light OS SWL-OS-DMF-8.6.x(0), 2023-12-07.09:17-42d8658
Linux A-7050SX3-48YC8 4.19.296-OpenNetworkLinux #1 SMP Thu Dec 7 09:28:30 UTC 2023 x86_64
SwitchLight ZTN Manual Configuration. Type help or ? to list commands.
(ztn-config) debug bash
*****************************WARNING******************************
Any/All activities within bash mode are UNSUPPORTED
This is intended ONLY for additional debugging ONLY by Arista TAC.
Please type "exit" or Ctrl-D to return to the CLI
*****************************WARNING******************************
root@A-7050SX3-48YC8:~# ofad-ctl gt PORT_CHANNEL_ENHANCED_HASH_SEED
GENTABLE : port_channel_enhanced_hash_seed
GENTABLE ID : 0x0006
hash_seed_valid: true
HASH_SEED1=0x000004d2
HASH_SEED2=0x000004d2
computed hash algorithm from seed: CRC32LO
root@A-7050SX3-48YC8:~#
Pseudo Multi-Chassis Link Aggregation
Currently in DMF, we support Link Aggregation Groups (LAGs) that allow 2 or more physical interfaces on the same DMF switch to be aggregated into 1 logical interface to increase the aggregate bandwidth and provide link redundancy against link failure. This feature works well if all the tools are connected to the same DMF delivery switch, which is typically the case when customer tools are co-located in the same physical location. However, in cases where tools are located in different data centers or physical locations, where a single DMF switch cannot connect to all the tools, load-balancing across two DMF delivery switches is required.
MLAG Components
- MLAG Domain: An MLAG domain is a logical grouping of two delivery switches that will participate in an MLAG.
- Peer Switch: Member switches added into the MLAG domain.
- MLAG Interface: An MLAG interface, configured under the MLAG domain, is a logical binding of two physical interfaces or LAG interfaces, one from each peer switch.
- Core MLAG Link: A fabric-facing MLAG link. A core switch LAG interface, whose members connect to the two peer switches participating in the MLAG domain.
- Delivery MLAG Link: An MLAG interface that is assigned the delivery interface role. This interface is used in a policy as a delivery interface.
- MLAG Member Interface: A physical interface or a LAG interface added into an MLAG interface.
- DMF Policy: A user-configured DMF policy that contains at least one MLAG delivery interface.
- Dynamic MLAG Domain Policy: Dynamically configured policies that follow the naming convention
_mlag_<DMF- policy>_<DeliverySwitch>
. For one user-configured MLAG policy, a policy that uses at least one MLAG delivery interface, two dynamic MLAG domain policies are created, one for each peer switch.
MLAG Limitations
- An MLAG domain cannot have more than two switches.
- A switch can only be a part of one MLAG domain.
- An MLAG interface can only have two member interfaces.
- An MLAG interface can only have one interface (physical interface or LAG interface) from each peer switch.
- Tunnel interfaces are not supported as members in MLAG interface configuration.
Configuring an MLAG via the CLI
- The user-configured policy delivers traffic from the filter switch to the core switch LAG interface.
- Dynamic Policy 1 delivers traffic to delivery switch 1.
- Dynamic Policy 2 delivers traffic to delivery switch 2.
Below are the details for each policy:
- Filter Interface(s) section lists the filter interface configured for the policy, Policy-1.
- Core Interface(s) section lists the interfaces that connect the filter switch and the core switch selected for the policy.
- MLAG Core Interface(s) section displays the core LAG interface that hashes the traffic towards the peer switches.
- MLAG Delivery Interface(s) section lists the delivery MLAG interface members.
- Filter Interfaces(s) section lists the dynamically configured interface name on DeliverySwitch1 to which the core switch is connected.
- MLAG Delivery Interface(s) section lists the delivery MLAG interface member on DeliverySwitch1.
- Filter Interfaces(s) section lists the dynamically configured interface name on DeliverySwitch2 to which the core switch is connected.
- MLAG Delivery Interface(s) section lists the delivery MLAG interface member on DeliverySwitch2.
MLAG Link Discovery
Link Layer Discovery Protocol (LLDP) is used to discover MLAG links. When the DMF Controller receives an LLDP message, it looks for the switch and interface names. If the switch is a part of an MLAG domain, and the reported interface corresponds to the MLAG interface, then it is classified as an MLAG link.
Controller-1(config)# show link all link-type mlag-member
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Links ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Active State Src switchSrc IF Name Dst switchDst IF Name Link Type Since
-|------------|-----------------|-----------|-----------------|-----------|-----------|-----------------------|
1 active CoreSwitch-1ethernet10DeliverySwitch-1ethernet50mlag-member 2022-11-11 21:54:28 UTC
2 active CoreSwitch-1ethernet20DeliverySwitch-2ethernet50mlag-member 2022-11-11 21:54:28 UTC
3 active DeliverySwitch-1ethernet50CoreSwitch-1ethernet10mlag-member 2022-11-11 21:54:28 UTC
4 active DeliverySwitch-2ethernet50CoreSwitch-1ethernet20mlag-member 2022-11-11 21:54:13 UTC
Controller-1(config)#
Configure MLAG via GUI
To configure an MLAG domain from the GUI, go to the
tab.Click on Create MLAG Domain and enter the following:
- Domain Name: Enter the MLAG domain alias.
- Peer Switch 1: From the drop-down, select the first switch that will be participating in the MLAG domain.
- Peer Switch 2: From the drop-down, select the second switch that will be participating in the MLAG domain.
- MLAG Interfaces: Enter an alias for the fabric-facing MLAG interface. This interface connects the core switch to the peer switches in the MLAG domain.
- Peer Switch 1 and Peer Switch 2: After selecting peer switches under the domain name, the peer switches under the MLAG interface will automatically be selected.
- Interface 1: Select the member interface that connects the core switch to DeliverySwitch-1
- Interface 2: Select the member interface that connects the core switch to DeliverySwitch-2.
- MLAG Delivery Interfaces: Enter an alias for each MLAG delivery interface.
- DMF Interface Name: Enter the DMF interface name for the MLAG delivery interface. This alias will be used to identify the delivery interface while configuring the DMF policy.
- Strip VLAN on Egress: Select the strip VLAN configuration for the MLAG delivery interface
- Peer Switch 1 and Peer Switch 2: These will be automatically selected based on the peer switches selected under the domain name.
- Interface 1: Select the member interface on Peer Switch 1.
- Interface 2: Select the member interface on Peer Switch 2.
The above screenshot displays the MLAG domain status. Click on the + button to expand the MLAG domain configuration and status of each MLAG interface.
Create MLAG Policy from GUI
Select + Create Policy to add a new policy.
Click Add 1 Interface and enter the following to configure the policy association.
- Name: Assign a unique name to the policy.
- Description: A description for the policy.
- Action: Forward (default), None, Capture, or Drop.
- Priority: 100 (default), or enter a value.
- Scheduling: Automatic (default), Now, Set Time, or Set Delay.
- : Select the filter interface (traffic source) for the policy.
- : Specify the traffic rule for the policy.
Click Create Policy.
Viewing Policy Statistics in the GUI
Viewing MLAG Links in the GUI
The above screenshot shows the MLAG links established between the core switch and the peer switches that are part of the MLAG domain. The links are discovered via LLDP message exchange.
Using LAG Interfaces as Members in MLAG Interfaces
MLAG interface members can be physical interfaces or LAG interfaces to increase bandwidth. To add a LAG member to an MLAG interface, use the following procedure:
Overlapping Policies in LAGs
An overlapping policy is dynamically configured if two configured policies share at least one filter interface, and at least one of the delivery interfaces is different.
- Policy-1 is configured to use the filter interface Filter-1, and the delivery interface MLAG-Tool-1.
- Policy-2 is configured to use the filter interface Filter-1, and the delivery interface MLAG-Tool-2.
- The above two policies will result in a overlapping policy. An overlapping policy will be configured following the naming convention _Policy-1_o_Policy-2.
MLAG Dynamic Policy | Parent Policy | Delivery Switch/Peer switch |
---|---|---|
_mlag_Policy-1_DeliverySwitch-1 | Policy-1 | DeliverySwitch-1 |
_mlag_Policy-1_DeliverySwitch-2 | Policy-1 | DeliverySwitch-2 |
_mlag_Policy-2_DeliverySwitch-1 | Policy-2 | DeliverySwitch-1 |
_mlag_Policy-2_DeliverySwitch-2 | Policy-2 | DeliverySwitch-2 |
_mlag Policy-1_o_Policy-2_DeliverySwitch-1 | _Policy-1_o_Policy-2 | DeliverySwitch-1 |
_mlag Policy-1_o_Policy-2_DeliverySwitch-2 | _Policy-1_o_Policy-2 | DeliverySwitch-2 |
The following policies, Policy-1 and Policy-2, share the same filter interface, Filter-1, but they are configured to use different delivery interfaces, MLAG-Tool-1 and MLAG-Tool-2. No priority is configured; therefore, these policies will be using the same default priority.
policy Policy-1
action forward
delivery-interface MLAG-Tool-1
filter-interface Filter-1
1 match ip src-ip 200.200.0.0 255.255.255.0
policy Policy-2
action forward
delivery-interface MLAG-Tool-2
filter-interface Filter-1
1 match ip dst-ip 100.100.0.0 255.255.255.0
Controller-1(config)# show policy
# Policy NameActionRuntime Status Type Priority Overlap Priy Push VLAN Filter BW Delivery BW Post Match Filt Traff Del Traffic Services
-|----------------------|-------|--------------|----------|--------|------------|---------|---------|-----------|----------------------|----------|--------|
1 Policy-1 forward installedConfigured1000 125Gbps80Gbps314Mbps315Mbps
2 Policy-2 forward installedConfigured1000 325Gbps80Gbps314Mbps315Mbps
3 _Policy-1_o_Policy-2 forward installedDynamic 1001 525Gbps80Gbps314Mbps315Mbps
- Policy-1: User-configured policy to forward packets matching source IP 200.200.0.0/24 to MLAG-Tool-1.
- Policy-2: User-configured policy to forward packets matching destination IP 100.100.0.0/24 to MLAG-Tool-2.
- _Policy-1_o_Policy-2: A dynamically configured overlapping policy with higher Overlap Priority to ensure that if a packet matches rules from both the policies (source IP of 200.200.0.1 and destination IP of 100.100.0.1) it will be forwarded to both MLAG-Tool-1 and MLAG-Tool-2.
Following are the dynamic policies configured for each delivery switch in the MLAG Domain.
- _mlag_Policy-1_DeliverySwitch-1: MLAG dynamic policy for Policy-1 for DeliverySwitch-1.
- _mlag_Policy-1_DeliverySwitch-2: MLAG dynamic policy for Policy-1 for DeliverySwitch-2.
- _mlag_Policy-2_DeliverySwitch-1: MLAG dynamic policy for Policy-2 for DeliverySwitch-1.
- _mlag_Policy-2_DeliverySwitch-2: MLAG dynamic policy for Policy-2 for DeliverySwitch-2.
_Policy-1_o_Policy-2 Dynamic Policies
- _mlag Policy-1_o_Policy-2_DeliverySwitch-1: MLAG dynamic policy for overlapping policy for DeliverySwitch-1.
- _mlag Policy-1_o_Policy-2_DeliverySwitch-2: MLAG dynamic policy for overlapping policy for DeliverySwitch-2.