Multi-Chassis Link Aggregation

Arista switches support Multi-Chassis Link Aggregation (mlag) to logically aggregate ports across two switches. For example, two 10-gigabit Ethernet ports, one each from two mlag configured switches, can connect to two 10-gigabit ports on a host, switch, or network device to create a link that appears as a single 20-gigabit port. mlag-configured ports provide Layer 2 multipathing, increased bandwidth, higher availability, and other improvements on traditional active-passive or Spanning Tree governed infrastructures.

The Multi-Chassis Link Aggregation section contains these topics:

mlag Introduction

High availability data center topologies typically provide redundancy protection at the expense of over-subscription by connecting Top-Of-Rack (TOR) switches and servers to dual aggregation switches. In these topologies, Spanning Tree Protocol prevents network loops by blocking half of the links to the aggregation switches. This reduces the available bandwidth by 50%.

Deploying mlag removes over-subscription by configuring an mlag link between two aggregation switches to create a single logical switching instance that utilizes all connections to the switches. Interfaces on both devices participate in a distributed port channel, enabling all active paths to carry data traffic while maintaining the integrity of the Spanning Tree topology.

mlag provides these benefits:
  • Aggregates multiple Ethernet ports across two switches.
  • Provides higher bandwidth links as network traffic increases.
  • Utilizes bandwidth more efficiently with fewer links blocked by STP.
  • Connects to other switches and servers by static LAG or LACP without other proprietary protocols.
  • Supports normal STP operation to prevent loops.
  • Supports active-active Layer-2 redundancy.
Note: For information on enabling Precision Timing Protocol (PTP) on an mlag interface, see the Timing Protocols chapter.
Note: The global STP configuration is derived from the primary peer device while the secondary device parameters are ignored. When STP is disabled on the primary device, the secondary device will not contain any STP configuration information from the primary device. As a result, the secondary device will not be able to decide on the port roles or states, and will remain in the default state which is the discarding state. This is an expected behavior.
Note: It is highly recommended that both mlag peer switches are identical platforms and run identical eos images. Running different images/platform may result in a failure to form an association with the mlag peer or see discrepancy in behavior.

mlag Conceptual Overview

mlag Operation Process

A Multi-chassis Link Aggregation Group (mlag) is a pair of links that terminate on two cooperating switches and appear as an ordinary Link Aggregation Group (LAG). The cooperating switches are mlag peer switches and communicate through an interface called a peer link. While the peer link’s primary purpose is exchanging mlag control information between peer switches, it also carries data traffic from devices that are attached to only one mlag peer and have no alternative path. An mlag domain consists of the peer switches and the control links that connect the switches.

In the figure below, Switch A and Switch B are peer switches in the mlag domain and connect to each other through the peer link. Each peer switch uses the peer address to form and maintain the peer link.

The mlag domain ID is a text string configured in each peer switch. mlag switches use this string to identify their peers. The mlag System ID (MSI) is the mlag domain’s MAC address. The MSI is automatically derived when the mlag forms and does not match the bridge MAC address of either peer. Each peer uses the MSI in STP and LACP PDUs.

The topology shown below contains four mlags: one mlag connects each device to the mlag domain. Each peer switch connects to the four servers through mlag link interfaces.

In a conventional topology, with dually-attaching devices to multiple switches for redundancy, Spanning Tree Protocol (STP) blocks half of the switch-device links. In the mlag topology, STP does not block any portion because it views the mlag Domain as a single switch and each mlag as a single port. The mlag protocol facilitates the balancing of device traffic between the peer switches.

Figure 1. mlag Domain Topology
When mlag is disabled, peer switches revert to their independent state. mlag is disabled by any of the following:
  • mlag configuration changes.
  • The TCP connection breaks.
  • The peer-link or local-interface goes down.
  • A switch does not receive a response to a keep alive message from its peer within a specified period.

mlag Interoperability with Other Features

The following sections describe mlag interaction with other switch features.

VLANs

VLAN parameters must be configured identically on each peer for the LAGs comprising the peer link and mlags. These parameters include the switchport access VLAN, switchport mode, trunk-allowed VLANs, the trunk native VLAN, and switchport trunk groups.

Configuration discrepancies may result in traffic loss in certain failure scenarios. Port-specific bridging configuration originates on the switch where the port is physically located.

LACP

Link Aggregation Control Protocol (LACP) should be used on all mlag interfaces, including the peer-link. LACP control packets reference the mlag system ID.

Static MAC Addresses

A static MAC address configured on an mlag interface is automatically configured on the peer’s corresponding interface. configuring static MAC addresses on both peers prevents undesired flooding if an mlag peer relationship fails.

If the mlag peering relationship is disabled, the static MAC previously learned from peer is removed.

Spanning Tree Protocol (STP)

When implementing mlag in a spanning tree network, spanning tree must be configured globally and on port-channels configured with an mlag ID. Port specific spanning tree configuration comes from the switch where the port physically resides. This includes spanning-tree PortFast BPDU Guard and BPDU filter.

Port Mirroring

A port channel which is a member of an mlag must not be used as the destination port for a port mirroring (port monitoring) session.

IPv6 Flow Label Hashing

Arista switches use the hashing algorithm to load-balance traffic among LAG members and Layer 3 ECMP (equal cost multipath) paths. For IP and IPv6 traffic, the hashing algorithm includes (if so configured for LAG) the IP packet fields such as source and destination IP addresses as well as source and destination ports for UDP and TCP traffic.

To improve traffic distribution for IPv6 traffic, IPv6 Flow Label field has been added to the hashing algorithm for both LAG and ECMP.

Note: IPv6 Flow Label is included in the LAG hashing algorithm only when the MAC header hashing is not enabled for IPv6 traffic.

mlag Maintenance

These sections describe tasks required for mlag to operate on the switch:

Ensuring Control Plane ACL Compatibility

The control plane Access Control List (ACL) on any interface participating in the mlag must be configured to allow only the peer link neighbor to generate mlag control traffic. The required rules are included in the default control plane ACL for Ethernet ports.

Any custom control plane ACL applied to a participating port must include these three rules:

permit tcp any any eq mlag ttl eq 255 
permit udp any any eq mlag ttl eq 255 
permit tcp any eq mlag any ttl eq 255

mlag peers that function as routers must each have routing enabled.

mlag Availability through a Single Functional Peer

mlag high availability advantages are fully realized when all devices that connect to one mlag switch also connect to the peer switch. A switch can continue supporting mlag when its peer is offline if the STP agent is restartable. When one peer is offline, data traffic flows from the devices through the mlag component link that connects to the functioning switch. When a switch is offline, its interfaces and ports do not appear in show mlag and show spanning tree protocol commands of the functioning peer.

To view the restartability status of the STP agent, use the detail option of theshow spanning-tree instancecommand:

switch-1# show spanning-tree instance detail | grep agent
Stp agent restartable                      :            True

STP agent restartability requires consistent configuration between the peers of STP, LACP, mlag, and switchport parameters. Events triggering an STP state machine change may also briefly prevent the STP agent from being restartable.

Reload Delay

If an mlag peer reboots, all ports except those in the peer-link port-channel remain in errdisabled state for a specified time, called the reload-delay period. This period allows all topology states to stabilize before the switch begins forwarding traffic. Each Arista switch defaults to the recommended reload-delay value, which varies by switch platform:
  • Fixed configuration switches: 300 seconds
  • Trident II modular switches: 1200 seconds
    • 7304
    • 7308
    • 7316
    • 7300X series
  • Sand platform fixed configuration switches: 600 seconds
    • 7280 series (except 7280CR2 and 7280SR2)
    • 7020 series
  • Sand platform modular switches: 1800 seconds
    • 7504
    • 7508
    • 7500E series
    • 7548S
  • Sand Jericho+ fixed configuration switches: 900 seconds
    • 7280CR2 series
    • 7280SR2 series

In those cases where network topology requires additional time to stabilize or where a shorter delay can be tolerated, the reload-delay period can be configured using the reload-delay mlag command.

Severing the physical connection (cable) that establishes the peer-link between mlag peers may result in a split brain state where each peer independently enters spanning tree state to prevent topology loops. Sessions established through one interface of a dual attached device may fail if its path is disrupted by the STP reconvergence, possibly resulting in temporarily lost connectivity. Sessions can be reestablished if permitted by the resulting topology.

Upgrading mlag Peers

mlag ISSU (In-Service Software Upgrade) upgrades eos software on one mlag peer with minimal traffic disruptions on active mlag interfaces and without changing the network topology.

Verifying Configuration Compatibility

A seamless eos upgrade on an mlag peer requires that the following features are configured consistently on each switch:
  • VLANs.
  • Switchport configuration on port channel interfaces that are configured with an mlag ID.
  • STP configuration (global).

Version Compatibility

A switch running mlag can be upgraded without disrupting mlag traffic when the upgraded eos version is compatible with the version on the peer switch. Refer to the Release Notes for a list of compatible eos versions.

Reload Warning Conditions

Entering an eos reload command while mlag is active generates warning messages if conditions that can result in packet loss during the upgrade are present. All warnings should be resolved before confirming the reload request. The following table displays the reload conditions and a common resolution method for each condition.

Table 1. Reload Warning Resolutions
Reload Condition Resolution Method
Compatibility check Refer to the Release Notes to verify that the new version is compatible with the currently installed version.
Active-partial mlag warning Bring up the remote port-channel. If the mlag is not actively used, then this warning can be ignored.
STP is not restartable Wait for STP to be restartable: typically 30 seconds, up to 120 seconds for a newly started STP agent. Refer to mlag Availability through a Single Functional Peer for information on checking restartability.
Reload delay too low Configure a reload delay value greater than or equal to the default. Recommend delay is 300 seconds for TOR switches, 900 seconds for modular switches, and 600 seconds for Sand platform fixed configuration switches (7020 and 7280 series (except 7280CR2 and 7280SR2)).
Peer has error-disabled interfaces Wait for reload-delay to expire on the peer.
Example
The following reload command generates mlag warning conditions that should be addressed before confirming the proceed with reload prompt. 
switch(config)# reload
If you are performing an upgrade, and the Release Notes for the
new version of eos indicate that mlag is not backwards-compatible 
with the currently installed version (4.9.2), the upgrade will 
result in packet loss.

The following mlags are not in Active mode. Traffic to or from these ports will 
be lost during the upgrade process.
                                                               local/remote 
  mlag      desc               state      local      remote          status 
--------- --------- ------------------- ---------- ----------- ------------ 
    14                active-partial       Po14        Po14         up/down 
    15                active-partial       Po15        Po15         up/down

Stp is not restartable. Topology changes will occur during the upgrade process.

The configured reload delay of 100 seconds is below the default
value of 300 seconds. A longer reload delay allows more time to
rollback an unsuccessful upgrade due to incompatibility.

The other mlag peer has errdisabled interfaces. Traffic loss will occur during 
the upgrade process.

Proceed with reload? [confirm]

Performing an mlag ISSU Upgrade

The following procedure performs an mlag ISSU upgrade:

  1. Verify configuration consistency on each peer (Verifying Configuration Compatibility).
  2. Verify version compatibility between the new and existing images (Version Compatibility).
  3. Configure reload-delay mlag (mlag Availability through a Single Functional Peer) if needed. Recommended delay period varies by switch type, and each switch defaults to its recommended delay period.
  4. Install the new image onto one of the peers:
    1. Upload the new image to the switch.
    2. Set the boot path to the new image.
    3. Enter the reload command.
  5. Resolve all reload warnings.
  6. Confirm the reload.
  7. Wait for mlag peers to renegotiate to the active state and reload-delay expiry on rebooted peer; until reload-delay period has expired, ports on the rebooted peer (except the peer-link) will be in errdisabled state with err-disabled reason being mlag-issu.

    Avoid configuration changes on both peers until after this step.

  8. Repeat the upgrade process for the other peer.

    When upgrading modular switches with dual supervisors, upgrade the standby supervisors first, then upgrade the active supervisors.

Forced Periodic ARP and NDP Refresh on IPv4 and IPv6 Networks

Overview

In mlag configurations where, each leaf connects to both spines (bowtie setups) and relies on Address Resolution Protocol (ARP) and Neighbor Discovery Protocol (NDP) refresh requests to keep VMACs in downstream mlag switches up to date. Configure forced periodic ARP and NDP refresh requests to send ARP and NDR in periodic intervals independently of the confirmed time of ARP and NDR entries in the kernel. By default, when various processes confirm a neighbor entry using ARP synchronization between mlag peers, the switch may not send an ARP refresh request for another duration of ARP aging timeout for IPv4 networks or NDP cache expiration time for IPv6 networks.

In topologies where the two upstream mlag switches with VXLAN virtual MAC addresses, the VMAC can age out from the MAC tables of the downstream mlag switches. Aging out can happen when the upstream switch sending APR or NDP requests through the downstream switch that owns the VMAC does not send an ARP or NDP request in that period because of ARP synchronization between the upstream mlag peers. Adding this feature ensures that the upstream switches send ARP and NDP requests at the configured cache interval to prevent aging out of the VMAC on downstream switches.

configuring mlag

These sections describe the basic mlag configuration steps:

configuring the mlag Peers

Connecting two switches as mlag peers requires the establishment of the peer link and an SVI that defines local and peer IP addresses on each switch.

The peer link is composed of a LAG between the switches. When all devices that connect to the mlag domain are dually connected to the switches through an mlag, a peer link of two Ethernet interfaces is sufficient to handle mlag control data and provide N+1 redundancy. When the domain connects to devices through only one mlag peer, the peer link may require additional Ethernet interfaces to manage data traffic.

Disruptions to peer link connectivity due to forwarding agent restarts may cause an extended mlag outage. Forwarding agent restart event include some configuration changes, such as port speed change or UFT mode change). The following precautions can reduce the risk of losing peer-link connectivity:
  • all switches: constructing peer-links from port-channels in preference to a single Ethernet interface.
  • modular systems: peer-link port-channel members should span multiple line cards.
  • multi-chip systems: peer-link port-channel member should span multiple chips.

Managing Switch Configuration Settings describes modular systems.

The steps that configure two switches as mlag peers include:

configuring the Port Channels, VLAN Interfaces, and IP addresses

The peer link is a normal port channel. The local address is the SVI that maps to the peer link port channel. The port channel and SVI must be configured on each peer switch. The port channel should be an active LACP port. The local and peer addresses must be located on the same IP address subnet. Autostate should be disabled on the SVI configured as the local interface.

Examples
  • These commands create an active mode LACP port channel interface from two Ethernet interfaces and configure it as part of a trunk group on each switch.

    The switchport mode trunk command permits all VLANs on the interface by default, so all VLANs are permitted on port channel 10 in the following example. The configuration of a trunk group for a VLAN restricts only that specific VLAN to the associated ports: VLAN 4094 is only permitted on port channel 10, and not on any other ports on the switch. It is important to remember that all VLANs must be permitted between the peers on the peer link for correct operation.

    Switch 1

    switch1# config
switch1(config)# vlan 4094
switch1(config-vlan-4094)# trunk group m1peer
switch1# config
switch1(config)# interface ethernet 1-2
switch1(config-if-et1-2)# channel-group 10 mode active
switch1(config-if-et1-2)# interface port-channel 10
switch1(config-if-po10)# switchport mode trunk
switch1(config-if-po10)# switchport trunk group m1peer
switch1(config-if-po10)# exit
switch1(config)#

    Switch 2

    switch2# config
switch2(config)# vlan 4094
switch2(config-vlan-4094)# trunk group m1peer
switch2(config-vlan-4094)# exit
switch2(config)# interface ethernet 1-2
switch2(config-if-et1-2)# channel-group 10 mode active
switch2(config-if-et1-2)# interface port-channel 10
switch2(config-if-po10)# switchport mode trunk
switch2(config-if-po10)# switchport trunk group m1peer
switch2(config-if-po10)# exit
switch2(config)#
  • These commands create an SVI for the local interface and associate it to the trunk group assigned to the peer link port channel.

    The SVI creates a Layer 3 endpoint in the switch and enables mlag processes to communicate via TCP. The IP address can be any unicast address that does not conflict with other SVIs. STP is disabled for the peer link vlan 4094 to prevent any potential STP disruption of inter peer communications. Recall that the VLAN has been restricted to port-channel 10 by the earlier trunk group configuration thus preventing potential Layer 2 loop conditions within vlan 4094.

    Switch 1

    switch1# config
switch1(config)# interface vlan 4094
switch1(config-if-vl4094)# ip address 10.0.0.1/30
switch1(config-if-vl4094)# no autostate
switch1(config-if-vl4094)# exit
switch1(config)# no spanning-tree vlan-id 4094
switch1(config)#

    Switch 2

    switch2# config
switch2(config)# interface vlan 4094
switch2(config-if-vl4094)# ip address 10.0.0.2/30
switch2(config-if-vl4094)# no autostate
switch2(config-if-vl4094)# exit
switch2(config)# no spanning-tree vlan-id 4094
switch2(config)#

configuring Peer Parameters

Peer connection parameters configure the connection between the mlag peer switches. This section describes the following peer configuration parameters.
mlag Configuration Mode

 

Peer connection parameters are configured in mlag-configuration mode. The mlag configuration (global configuration) command places the switch in mlag configuration mode.

Example

This command places the switch in mlag configuration mode.

switch(config)# mlag configuration
switch(config-mlag)#
Local VLAN Interface

 

The local interface specifies the SVI upon which the switch sends mlag control traffic. The local IP address is specified within the definition of the VLAN associated with the local interface. The Peer Address configures the control traffic destination on the peer switch.

The local-interface command specifies a VLAN interface as the peer link SVI.

Example

This command configures vlan 4094 as the local interface.

switch(config-mlag)#local-interface vlan 4094
switch(config-mlag)#
Peer Address

 

The peer address is the destination address on the peer switch for mlag control traffic. If the peer IP address is unreachable, mlag peering fails and both peer switches revert to their independent state.

The peer-address command specifies the peer address.

Example

This command configures a peer address of 10.0.0.2.

switch(config-mlag)# peer-address 10.0.0.2
switch(config-mlag)#
Domain ID

 

The mlag domain ID is a unique identifier for an mlag domain. The mlag domain ID must be the identical on each switch to facilitate mlag communication.

The domain-id command configures the mlag domain ID.

Example

This command configures mlagDomain as the domain ID:

switch(config-mlag)# domain-id mlagDomain
switch(config-mlag)#
Heartbeat Interval and Timeout

 

The heartbeat interval specifies the period between the transmission of successive keepalive messages. Each mlag switch transmits keepalive messages and monitors message reception from its peer. The heartbeat timeout is reset when the switch receives a keepalive message. If the heartbeat timeout expires, the switch disables mlag under the premise that the peer switch is not functioning.

The heartbeat-interval (mlag) command configures the heartbeat interval between 1 and 30 seconds, with a default value of 4 seconds. The heartbeat timeout expiry is 30 seconds.

Note: On 7500 and 7500E Series Switches, Arista recommends setting the heartbeat interval to 10 seconds.

Example

This command configures the heartbeat interval as 2500 milliseconds (2.5 seconds).

switch(config-mlag)# heartbeat-interval 2500
switch(config-mlag)#
Reload Delay Period

 

The reload delay period specifies the interval that non-peer links are disabled after an mlag peer reboots. This interval allows non-peer links to learn multicast and OSPF states and synchronize ARP caches before the ports start handling traffic. Each Arista switch defaults to the recommended reload-delay value, which varies by switch platform.
  • Fixed configuration switches: 300 seconds (five minutes).
  • Trident II platform modular switches: 1200 seconds (twenty minutes).
  • Sand platform fixed configuration switches(7020 and 7280 series (except 7280CR2 and 7280SR2)): 600 seconds (ten minutes).
  • Sand platform modular switches: 1800 seconds (thirty minutes).

In those cases where network topology requires additional time to stabilize or where a shorter delay can be tolerated, the reload-delay period can be configured using the reload-delay mlag command.

Example

This command configures the reload delay interval as 2.5 minutes (150 seconds).

switch(config-mlag)# reload-delay 150
switch(config-mlag)#
Shutdown

 

The shutdown (mlag) command disables mlag operations without disrupting the mlag configuration. The no mlag configuration command (global configuration mode) disables mlag and removes the mlag configuration. The no shutdown command resumes mlag activity.

Examples
  • This command disables mlag activity on the switch. 
    switch(config-mlag)# shutdown
switch(config-mlag)#
  • This command resumes mlag activity on the switch. 
    switch(config-mlag)# no shutdown
switch(config-mlag)#

configuring mlag Peer Gateway

In an mlag setup, routing on a mlag peer switch is possible using its own bridge system MAC, VARP MAC, or VRRP MAC. On a peer receiving an IP packet with destination MAC set to one of these MACs, a packet gets routed if its hardware has enough information to route the packet. configuring sending traffic to a cached MAC involves routing the session table and mlag peer traffic if packets are received with the MAC peer.

Examples
  • This command enables the mlag peer gateway. 
    switch(config)# ip virtual-router mac-address mlag-peer
switch1(config)#
  • This command disables the mlag peer gateway. 
    switch(config)# no ip virtual-router mac-address mlag-peer
switch1(config)#

configuring Ingress Replication to LAGs

Hardware support for ingress replication to LAGs is enabled by default when the user configures ingress replication. When multicast traffic is sent over the LAG, the hardware uses its built-in algorithm, based on the L2/L3/L4 headers, to load balance traffic over ports in the LAG. When a port goes down in a LAG, the hardware quickly hashes the multicast traffic over the remaining ports in the LAG, resulting in fewer drops than software based LAG support.

Examples
  • This command enables ingress replication. 
    switch(config)# platform sand multicast replication default ingress
switch(config)#
  • This command configures the maximum members (within a range of 1 through 64) for ingress only replication in a multicast group. 
    switch(config)# platform sand multicast replication ingress maximum 32
switch(config)#

configuring mlag Services

An mlag is a pair of links that originate on a network attached device and terminate on the two mlag peer switches. The mlag switches coordinate traffic to the device through a common mlag (port-channel interface configuration) command on the interfaces that connect to the device.

The mlag ID differs from the mlag domain ID. The mlag domain ID is assigned globally per switch in mlag configuration mode, and the same mlag domain ID must be on both switches.

Port channels configured as an mlag must have identical port channel numbers. Although the mlag ID is a distinct parameter from the port channel number, best practices recommend assigning the mlag ID to match the port channel number.
Note: Arista recommends configuring the downstream switch or router connected to the mlag peers to negotiate a LAG using LACP rather than configuring static LAGs. (On Arista switches, LACP is enabled using the command channel-group mode active. See Port Channels and LACP.) Using LAGs negotiated by LACP can help avoid problems caused by miscabling because LACP PDUs include the system ID and LACP key, which identify the switch and the LAG it belongs to. If mlag miscabling happens using a LACP LAG, some ports become inactive in the LAG which prevents it from bridging traffic erroneously. More importantly, when the mlag state is primary on one switch and inactive on another, the peer switches are acting as two independent L2 switches rather than as one logical L2 switch. In this scenario, the downstream switch will have its LAG (logical L2 port) connected to two distinct L2 switches, which can result in spanning tree problems on the downstream switch because it will receive BPDUs with different system IDs on the same LAG. If spanning tree is not running, an L2 loop will be created between mlag peer switches and the downstream device. Allowing LACP to negotiate the downstream LAGavoids this problem because only ports connected to a given peer become active in the LAG, and the other set of ports cannot bridge traffic.

The example below does not follow this convention to emphasize the distinct parameters. The example in mlag Implementation Example follows the best practices convention.

Examples

  • These switch1 commands bundle Ethernet interfaces 3 and 4 in port channel 20, then associate that port channel with mlag 12. 
    switch1(config)# interface ethernet 3-4
switch1(config-if-et3-4)# channel-group 20 mode active
switch1(config-if-et3-4)# interface port-channel 20
switch1(config-if-po20)# mlag 12
switch1(config-if-po20)# exit
switch1(config)#
  • These switch2 commands bundle Ethernet interfaces 9 and 10 in port channel 20, then associate that port channel with mlag 12. 
    switch2(config)# interface ethernet 9-10
switch2(config-if-et9-10)# channel-group 20 mode active
switch2(config-if-et9-10)# interface port-channel 20
switch2(config-if-po20)# mlag 12
switch2(config-if-po20)# exit
switch2(config)#
  • These commands configure the port channels that attach to the mlag on network attached device: 
    NAD(config)# interface ethernet 1-4
NAD(config-if-Et1-4)# channel-group 1 mode active
NAD(config-if-Et1-4)# exit
NAD(config)#

The following figure displays the result of the interface mlag configuration.

Figure 2. mlag Interface Configuration

EVPN - mlag Single-homed Hosts

Add a local VTEP to each mlag peer for the control plane to advertise singly connected hosts directly behind a specific local VTEP-mlag peer. Outgoing EVPN route advertisements contain nexthop and router MAC extended community when multi-VTEP mlag mode is enabled. The following table summarizes the information. For symmetric IRB support, mlag shared router MAC must be enabled.

Table 2. Route Advertisement
Routes Single-homed Host Multi-homed Host
MAC/IP routes (Type-2) Local VTEP IP / Bridge MAC mlag VTEP IP / Shared mlag Router MAC
IMET routes (Type-3) mlag VTEP IP
IP Prefix routes (Type-5) Local VTEP IP / Bridge MAC

In multi-VTEP mlag mode, IP prefix routes are advertised independently by each mlag peer with its own local VTEP IP as nexthop, even when both peers are connected to that route. Additionally, egress VXLAN packets use the appropriate source IP address to match what is advertised by the EVPN control plane.

Displaying EVPN with mlag Single-home Hosts

The following displays the nexthop address of locally generated EVPN Type-2 and Type-3 routes. The real IP address helps identify the VTEP in use. The nexthop of locally generated EVPN Type-5 routes are displayed as "-". The shared mlag VTEP IP is 1.1.1.1 and the local VTEP IPs of the two mlag peers are 1.0.1.1 and 1.0.2.2.

switch# show bgp evpn
BGP routing table information for VRF default
Router identifier 0.0.0.1, local AS number 300
Route status codes: s - suppressed, * - valid, > - active, # - not installed, E - ECMP head, e - ECMP
                    S - Stale, c - Contributing to ECMP, b - backup
                    % - Pending BGP convergence
Origin codes: i - IGP, e - EGP, ? - incomplete
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

Network	Next Hop		Metric	LocPref	Weight	Path
* >	RD: 1.0.1.1:200 mac-ip 52de.3c26.a0b0 10.2.0.2
                1.0.1.1		-		-	    0		i
* >	RD: 1.0.1.1:100 mac-ip 822c.0630.7ef4 10.1.0.2
                1.1.1.1 		-		-	   0		 i
	RD: 1.0.2.2:100 mac-ip 822c.0630.7ef4 10.1.0.2
                1.1.1.1 		-		100	 0		 i
* >	RD: 1.0.2.2:200 mac-ip 8650.1ecc.3595 10.2.0.3
                1.0.2.2 		-		100	  0		i
* >	RD: 1.0.1.1:100 imet 1.1.1.1
                1.1.1.1 		-		-	    0		i
* >	RD: 1.0.1.1:200 imet 1.1.1.1
                1.1.1.1 		-		-	     0	      i
* >	RD: 31000:300 ip-prefix 10.1.0.0/16
                -	 		-		-	    0		i
* 	RD: 31000:300 ip-prefix 10.1.0.0/16
                1.0.2.2	 	-		100         0		i

Configuration Considerations

  • The mlag VTEP IP advertises BUM traffic. VXLAN flooded packets may be seen on the peer-link.
  • The singly connected interfaces or non-mlag interfaces must be placed in an exclusive VLAN with an mlag interface as a member.
  • Enable the mlag shared router MAC to support symmetric IRB.
  • For an EVPN Type-5 setup, the VRF to VNI mapping must be configured on both VTEPs of the mlag pair.
  • For an EVPN Type-5 setup as well as EVPN symmetric IRB setup, the VRF to VNI mapping must be configured identically across all the VTEPs in the network. The same VRF must be mapped to the same value of VNI on all the VTEPs that participate in the EVPN network.

configuring mlag Layer 2 Subinterfaces

To configure an mlag L2 subinterface, first configure an mlag interface, and then add an L2 subinterface to the interface on both peers. Once configured, an mlag subinterface automatically inherits the mlag ID of the parent interface to generate a unique mlag ID.

For example, if configuring Port-Channel10 as mlag 11, then the subinterface has the mlag ID 11.1 as an implicit configuration.

When configuring an L2 subinterface, the parent interface must be a routed port. This does not comply with mlag protocols as the mlag interface must not be a routed port. An interface with the parameters, mlag 10 and no switchport, becomes ignored as an mlag interface until you create L2 subinterfaces on that interface.

Example Configuration

The following commands add an mlag L2 subinterface to switch1 and switch2:

switch1#conf
switch1(config)#interface ethernet 3-4
switch1(config-if-et3-4)#channel-group 10 mode active
switch1(config-if-et3-4)#interface port-channel 10
switch1(config-if-po10)#mlag 10
switch1(config-if-po10)#no switchport
switch1(config-if-po10)#interface port-channel 10.1
switch1(config-if-po10.1)#encapsulation dot1q vlan 10
switch1(config-if-po10.1)#vlan id 100
switch1(config-if-po10.1)#exit
switch1(config)#
switch2#conf
switch2(config)#interface ethernet 9-10
switch2(config-if-et9-10)#channel-group 10 mode active
switch2(config-if-et9-10)#interface port-channel 10
switch2(config-if-po10)#mlag 10
switch2(config-if-po10)#no switchport
switch2(config-if-po10)#interface port-channel 10.1
switch2(config-if-po10.1)#encapsulation dot1q vlan 10
switch2(config-if-po10.1)#vlan id 100
switch2(config-if-po10.1)#exit
switch2(config)#

To display information about mlag L2 subinterfaces configured on a switch, use the show mlag subinterfaces command:

switch#show mlag subinterfaces
mlag       Desc        Local Sub-Interface    Local Status
---------- ---------- ---------------------  -------------
10.1                           Po10.1             up
10.2                           Po10.2             up
10.3                           Po10.3             up
10.4                           Po10.4             up

If you have an mlag interface with a routed port, but no subinterfaces configured, you may see output similar to the following:

switch(config)#show mlag interfaces
            
mlag       desc                 state       local       remote          status
---------- ---------- --------------------- ----------- ------------ ------------
10                             active-full       Po10         Po10           up/up
11                       disabled-ignored      Po11            -               -
12                             active-full       Po12         Po12           up/up

mlag interface 11 has a routed port but no subinterfaces configured and returns the state disabled-ignored.

configuring Forced Periodic ARP and NDP Refresh

To enable Forced Periodic ARP Refresh on an IPv4 network, use the following command:

switch(config-if-Et1)#arp aging refresh always

To enable Forced Periodic NDP Refresh on an IPv6 network, use the following command:

switch(config-if-Et1)#ipv6 nd cache refresh always

To disable the feature, use the following command with no or default prefix:

switch(config-if-Et1)#no arp aging refresh always
switch(config-if-Et1)#no ipv6 nd cache refresh always

mlag Implementation Example

This example creates an mlag Domain, then configures mlag connections between the peer switches and four Network Attached Devices (NADs). The mlag switches connect through a LAG and communicate with the NADs through mlags. Although the NADs can be any device that supports LACP LAGs, the devices in this example are Arista switches.

Figure 3. mlag Implementation Example

Topology

Figure 3 displays the mlag topology. Switch 1 and Switch 2 are mlag peers that logically represent a single Layer 2 switch. The peer link between the switches contains the following interfaces:
  • Switch 1: ethernet 47, ethernet 48
  • Switch 2: ethernet 23, ethernet 24

The example configures mlags from the mlag Domain to four network attached devices (NAD-1, NAD-2, NAD-3, NAD-4).

configuring the Peer Switch Connections

To configure the switches in the described topology, perform the tasks in these sections:

configuring the Peer Switch Port Channels

These commands create the port channels the switches use to establish the peer link.

These commands create port channels on Switch1
switch1(config)# interface ethernet 47-48
switch1(config-if-et47-48)# channel-group 101 mode active
switch1(config-if-et47-48)# interface port-channel 101
switch1(config-if-po101)# switchport mode trunk
switch1(config-if-po101)# switchport trunk group peertrunk
switch1(config-if-po101)# exit
switch1(config)#
These commands create port channels on Switch2
switch2(config)# interface ethernet 23-24
switch2(config-if-et23-24)# channel-group 201 mode active
switch2(config-if-et23-24)# interface port-channel 201
switch2(config-if-po201)# switchport mode trunk
switch2(config-if-po201)# switchport trunk group peertrunk
switch2(config-if-po201)# exit
switch2(config)#

configuring the Peer Switch SVIs

For each peer switch, these commands create an SVI and associate it to the trunk group assigned to the peer link port channel. STP is disabled on the VLAN.

These commands configure the SVI on Switch1
switch1(config)# vlan 4094
switch1(config-vlan-4094)# trunk group peertrunk
switch1(config-vlan-4094)# interface vlan 4094
switch1(config-if-vl4094)# ip address 172.17.0.1/30
switch1(config-if-vl4094)# no autostate
switch1(config-if-vl4094)# exit
switch1(config)# no spanning-tree vlan-id 4094
switch1(config)#
These commands configure the SVI on Switch2
switch2(config)# vlan 4094
switch2(config-vlan-4094)# trunk group trunkpeer
switch2(config-vlan-4094)# interface vlan 4094
switch2(config-if-vl4094)# ip address 172.17.0.2/30
switch2(config-if-vl4094)# no autostate
switch2(config-if-vl4094)# exit
switch2(config)# no spanning-tree vlan-id 4094
switch2(config)#

configuring the Peer Links

These commands create the peer links on each mlag switch.

These commands create peer links on Switch1
switch1(config)# mlag configuration            
switch1(config-mlag)# local-interface vlan 4094          
switch1(config-mlag)# peer-address 172.17.0.2      
switch1(config-mlag)# peer-link port-channel 101            
switch1(config-mlag)# domain-id mlag_01               
switch1(config-mlag)# heartbeat-interval 2500            
switch1(config-mlag)# reload-delay 150             
switch1(config-mlag)# exit            
switch2(config)#
These commands create peer links on Switch2
switch2(config)# mlag configuration       
switch2(config-mlag)# local-interface vlan 4094           
switch2(config-mlag)# peer-address 172.17.0.1           
switch2(config-mlag)# peer-link port-channel 201                 
switch2(config-mlag)# domain-id mlag_01            
switch2(config-mlag)# heartbeat-interval 2500          
switch2(config-mlag)# reload-delay 150       
switch2(config-mlag)# exit            
switch2(config)#

configuring Peer Switch mlags

These commands create the mlags that connect the mlag domain to the network attached devices.

These commands configure mlag 1 on Switch1

switch1(config)# interface ethernet 17-18
switch1(config-if-et17-18)# channel-group 1 mode active
switch1(config-if-et17-18)# interface port-channel 1
switch1(config-if-po1)# mlag 1
switch1(config-if-po1)# exit
switch1(config)#

These commands configure mlag 1 on Switch2

switch2(config)# interface ethernet 1-2
switch2(config-if-et1-2)# channel-group 1 mode active
switch2(config-if-et1-2)# interface port-channel 1
switch2(config-if-po1)# mlag 1
switch2(config-if-po1)# exit
switch2(config)#

These commands configure mlag 2 on Switch1

switch1(config)# interface ethernet 19-20
switch1(config-if-et19-20)# channel-group 2 mode active
switch1(config-if-et19-20)# interface port-channel 2
switch1(config-if-po2)# mlag 2
switch1(config-if-po2)# exit
switch1(config)#

These commands configure mlag 2 on Switch2

switch2(config)# interface ethernet 3-4
switch2(config-if-et3-4)# channel-group 2 mode active
switch2(config-if-et3-4)# interface port-channel 2
switch2(config-if-po2)# mlag 2
switch2(config-if-po2)# exit
switch2(config)#

These commands configure mlag 3 on Switch1

switch1(config)#  interface ethernet 23
switch1(config-if-et23)# channel-group 3 mode active
switch1(config-if-et23)# interface port-channel 3
switch1(config-if-po3)# mlag 3
switch1(config-if-po3)# exit
switch1(config)#

These commands configure mlag 3 on Switch2

switch2(config)# interface ethernet 7
switch2(config-if-et7)# channel-group 3 mode active
switch2(config-if-et7)# interface port-channel 3
switch2(config-if-po3)# mlag 3
switch2(config-if-po3)# exit
switch2(config)#

These commands configure mlag 4 on Switch1

switch1(config)# interface ethernet 25
switch1(config-if-et25)# channel-group 4 mode active
switch1(config-if-et25)# interface port-channel 4
switch1(config-if-po4)# mlag 4
switch1(config-if-po4)# exit
switch1(config)#

These commands configure mlag 4 on Switch2

switch2(config)# interface ethernet 9
switch2(config-if-et9)# channel-group 4 mode active
switch2(config-if-et9)# interface port-channel 4
switch2(config-if-po4)# mlag 4
switch2(config-if-po4)# exit
switch2(config)#

configuring the Network Attached Devices

These commands create the LAGs on the Network Attached Devices that connect to the mlag domain.

These commands configure the port channels on NAD-1

NAD-1(config)# interface ethernet 7-10
NAD-1(config-if-Et7-10)# channel-group 1 mode active
NAD-1(config-if-Et7-10)# exit
NAD-1(config)#

These commands configure the port channels on NAD-2

NAD-2(config)# interface ethernet 25-28
NAD-2(config-if-Et25-28)# channel-group 7 mode active
NAD-2(config-if-Et25-28)# exit
NAD-2(config)#

These commands configure the port channels on NAD-3

NAD-3(config)# interface ethernet 3-4
NAD-3(config-if-Et3-4)# channel-group 5 mode active
NAD-3(config-if-Et3-4)# exit
NAD-3(config)#

These commands configure the port channels on NAD-4

NAD-4(config)# interface ethernet 1-2
NAD-4(config-if-Et1-2)# channel-group 2 mode active
NAD-4(config-if-Et1-2)# exit
NAD-4(config)#

Verification

The following tasks verify the mlag peer and connection configuration:

Verify the Peer Switch Connection

To display the mlag configuration and the mlag status on switch 1, use the show mlag command:

switch1# show mlag    
mlag Configuration:
domain-id         :            mlag_01
local-interface   :           Vlan4094
peer-address      :         172.17.0.2
peer-link         :    Port-Channel101

mlag Status:
state             :             Active
peer-link status  :                 Up
local-int status  :                 Up
system-id         :  02:1c:FF:00:15:38

mlag Ports:
Disabled          :   0
Configured        :   0
Inactive          :   0
Active-partial    :   0
Active-full       :   4

To display the mlag configuration and the mlag status on switch 2, use the show mlag command:

switch2# show mlag
mlag Configuration:
domain-id         :            mlag_01
local-interface   :           Vlan4094
peer-address      :         172.17.0.1
peer-link         :    Port-Channel102

mlag Status:
state             :             Active
peer-link status  :                 Up
local-int status  :                 Up
system-id         :  02:1c:FF:00:15:41

mlag Ports:
Disabled          :   0
Configured        :   0
Inactive          :   0
Active-partial    :   0
Active-full       :   4

Verify the mlags

The show mlag interfaces command displays mlag connections between the mlag switches and the Network Attached Devices.
  • This show mlag interfaces command displays mlag connections between the mlag peer switch 1 and the network attached devices: 
    switch1# show mlag interfaces
                                                               local/remote
  mlag     desc                   state     local     remote         status
----------------------------------------------------------------------------
     1     sw1.po1          active-full       Po1        Po1          up/up
     2     sw1.po2          active-full       Po2        Po2          up/up
     3     sw1.po3          active-full       Po3        Po3          up/up
     4     sw1.po4          active-full       Po4        Po4          up/up
  • The following show mlag interfaces command, with the detail option, displays mlag connections between the mlag peer switch 1 and the network attached devices. 
    switch2#show mlag interfaces detail
                               local/remote
mlag       state local remote   oper   config          last change  changes
----------------------------------------------------------------------------
   1 active-full   Po1    Po1  up/up  ena/ena  6 days, 2:08:28 ago        5
   2 active-full   Po2    Po2  up/up  ena/ena  6 days, 2:08:30 ago        5
   3 active-full   Po3    Po3  up/up  ena/ena  6 days, 2:08:33 ago        5
   4 active-full   Po4    Po4  up/up  ena/ena  6 days, 2:08:41 ago        5
switch2#

Verify Spanning Tree Protocol (STP)

STP functions can be displayed from each peer switch. mlag interfaces are displayed as a single entry. Configured interfaces on each switch that are not included in an mlag are displayed. Local interfaces have the normal notation; remote interfaces are preceded by P or Peer.

VLAN Output 1: Assume VLAN 3903 includes mlag 1
switch1# show spanning-tree vlan-id 3903
Spanning tree instance for vlan 3903
VL3903
  Spanning tree enabled protocol rapid-pvst
  Root ID    Priority    36671
             Address     001c.730c.3009
             Cost        1999 (Ext) 0 (Int)
             Port        105 (Port-Channel5)
             Hello Time  2.000 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    36671  (priority 32768 sys-id-ext 3903)
             Address     021c.7300.1319
             Hello Time  2.000 sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role       State      Cost      Prio.Nbr Type
---------------- ---------- ---------- --------- -------- --------------------
Po1              root       forwarding 1999      128.105  P2p
switch1#

The output displays mlag 1 under its local interface name (Po1). A peer interface is not displayed because spanning tree considers the local and remote Port Channels as a single mlag interface.

VLAN Output 2: Assume VLAN 3908 does not include any mlags
switch1# show spanning-tree vlan-id 3908
Spanning tree instance for vlan 3908
VL3908
  Spanning tree enabled protocol rapid-pvst
  Root ID    Priority    36676
             Address     021c.7300.1319
             This bridge is the root

  Bridge ID  Priority    36676  (priority 32768 sys-id-ext 3908)
             Address     021c.7300.1319
             Hello Time  2.000 sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role       State      Cost      Prio.Nbr Type
---------------- ---------- ---------- --------- -------- ---------------
Et17             designated forwarding 2000      128.217  P2p
Et18             designated forwarding 2000      128.218  P2p
PEt17            designated forwarding 2000      128.17   P2p
PEt18            designated forwarding 2000      128.18   P2p

The output displays all interfaces from both switches. Each interface is explicitly displayed because they are individual units that STP must consider when selecting ports to block.

  • Et17 and Et18 are located on the switch where the show spanning-tree command is issued.
  • PEt17 and PEt18 are located on the remote switch from where the command was issued

An identical command issued on the peer switch displays similar information.

Verify the mlag does not create topology loops (show spanning-tree blocked)
switch1# show spanning-tree blocked
Name       Blocked Interfaces List
------------------------------------------------------

Number of blocked ports (segments) in the system : 0
switch1#

Verify the mlag Port Channel

Issue the command show port-channel for channels 1-4 from switch 1:

switch# show port-channel 1-4
Port Channel Port-Channel1:
  Active Ports: Ethernet17     Ethernet18     PeerEthernet1 PeerEthernet2
Port Channel Port-Channel2:
  Active Ports: Ethernet19     Ethernet20     Ethernet21     Ethernet22
                PeerEthernet3 PeerEthernet4 PeerEthernet5 PeerEthernet6
Port Channel Port-Channel3:
  Active Ports: Ethernet23     Ethernet24     PeerEthernet7 PeerEthernet8
Port Channel Port-Channel4:
  Active Ports: Ethernet25     Ethernet26     PeerEthernet9 PeerEthernet10

Issue the command show port-channel load-balance fields detailed command for channel 1 from switch 2:

switch2# show port-channel 1 detailed
Port Channel Port-Channel1:
  Active Ports:
       Port                Time became active       Protocol    Mode
    -----------------------------------------------------------------------
       Ethernet17          7/7/11 15:27:36          LACP        Active
       Ethernet18          7/7/11 15:27:36          LACP        Active
       PeerEthernet1       7/7/11 15:27:36          LACP        Active
       PeerEthernet2       7/7/11 15:27:36          LACP        Active

Verify the VLAN Membership

The show vlancommand displays VLAN member ports, including mlag ports and ports on each peer not bundled in an mlag.

Switch1# show vlan 3903, 3908
VLAN  Name                             Status    Ports
----- -------------------------------- --------- -------------------------------
3903  ar.mg.rn.172.17.254.16/29        active    Cpu, Po1
3908  po.ra.ar.mg.172.17.254.64/29     active    Cpu, Et17, Et18, PEt17, PEt18

mlag Commands

Global mlag Commands

mlag Interface Configuration Commands

mlag Configuration Commands

Display Commands

arp aging refresh always

The arp aging refresh always command enables the switch to send forced periodic ARP requests for an mlag configuration in an IPv4 network. This may be necessary to prevent the VMAC from aging out on downstream switches. Using the no or default disables the feature.

Command Mode

Interface Configuration Mode

Command Syntax

arp aging refresh always

no arp aging refresh always

default arp aging refresh always

Example

Use the following command to enable Forced Periodic ARP requests on the switch:

switch(config-if-Et1)#arp aging refresh always

domain-id

The domain-id command specifies a name for the Multi-chassis Link AGgregation (mlag) domain.

The no domain-id and default domain-id commands remove the mlag domain name by deleting the domain-id statement from running-config.

Command Mode

mlag Configuration

Command Syntax

domain-id identifier

no domain-id

default domain-id

Parameters

identifier     alphanumeric string that names the mlag domain.

Example

This command names the mlag domain mlag1.

switch(config)# mlag
switch(config-mlag)# domain-id mlag1
switch(config-mlag)#

dual-primary detection delay

Use the dual-primary detection delay command to configure a dual primary detection delay with an optional action to errdisable all interfaces on secondary mlag peer after a dual primary condition is detected.

Command Mode

mlag configuration mode

Command Syntax

dual-primary detection delay seconds [action erridisable all-interfaces]

no dual-primary detection delay seconds [action erridisable all-interfaces]

default dual-primary detection delay seconds [action erridisable all-interfaces]

Parameters
  • seconds Dual primary detection delay in seconds.
  • actionSpecifies the action when dual-primary is detected.
  • erridisableErrdisable interfaces.
  • all-interfacesDisables all Ethernet interfaces except the peer-link.
Examples
  • In this example, the command errdisables all physical interfaces on secondary mlag peer when dual primary condition is detected. 
    switch(config-mlag)# dual-primary detection delay 5 action errdisable all-interfaces

    Both mlag peers must have equivalent configurations.

  • The following command removes the Dual Primary Detection feature. You must unconfigure both mlag peers. 
    switch(config-mlag)# no dual-primary detection

dual-primary recovery delay

Use the dual-primary recovery delay command to configure dual-primary detection recovery-delay for mlag interfaces and non-mlag interfaces. Negating the configurations or configuring default values makes both recovery delay values reset back to 0. The non-mlag delay must always be less than the mlag delay so you have more time for L3 convergence before enabling mlag interfaces.

Command Mode

mlag configuration mode

Command Syntax

dual-primary recovery delay mlag seconds non-mlag seconds

no dual-primary recovery delay mlag seconds non-mlag seconds

default dual-primary recovery delay mlag seconds non-mlag seconds

Parameters
  • mlag seconds Delay in seconds after dual-primary detection resolves until non peer-link ports that are part of an mlag are enabled. Range 0 - 1000 seconds. A suggested value for mlag is 60 seconds. These values can be adjusted depending on the network scale.
  • non-mlag seconds Delay in seconds after dual-primary detection resolves until ports that are not part of an mlag are enabled. Range 0 - 1000 seconds. A suggested value for non-mlag is 0 seconds. These values can be adjusted depending on the network scale.

Example

switch(config)#mlag
switch(config-mlag)#dual-primary recovery delay mlag 60 non-mlag 0

heartbeat-interval (mlag)

The heartbeat-interval command configures the interval at which heartbeat messages are issued in a Multi-chassis Link AGgregation (mlag) configuration.

The no heartbeat-interval and default heartbeat-interval commands revert the heartbeat interval to the default setting by removing the heartbeat-interval command from running-config.

Command Mode

mlag Configuration

Command Syntax

heartbeat-interval period

no heartbeat-interval

default heartbeat-interval

Parameters

period     Interval duration in milliseconds. Value ranges from 1000 through 30000 milliseconds. Default interval is 4000 milliseconds.

Guidelines

Heartbeat messages flow independently in both directions between the mlag peers. If a peer stops receiving heartbeat messages within the expected time frame (30 seconds), the other peer can assume it no longer functions and without intervention or repair, the mlag becomes disabled. Both switches revert to their independent state.

Note: On 7500 and 7500E Series Switches, Arista recommends setting the heartbeat interval to 10 seconds.

Example

This command configures the heartbeat interval to 15000 milliseconds: 
switch(config)# mlag
switch(config-mlag)# heartbeat-interval 15000
switch(config-mlag)#

ipv6 nd cache refresh always

The ipv6 nd cache refresh always command enables the switch to send forced periodic ARP requests for an mlag configuration in an IPv6 network. This may be necessary to prevent the VMAC from aging out on downstream switches. Using the no or default disables the feature.

Command Mode

Interface Configuration Mode

Command Syntax

ipv6 nd cache refresh always

no ipv6 nd cache refresh always

default ipv6 nd cache refresh always

Example

Use the following command to enable Forced Periodic ARP requests on the switch:

switch(config-if-Et1)#ipv6 nd cache refresh alwayss

local-interface

The local-interface command assigns a VLAN interface for use in Multi-chassis Link AGgregation (mlag) configurations. The VLAN interface is used for both directions of communication between the mlag peers.

The no local-interface and default local-interface commands delete the VLAN interface assignment by removing the local-interface command from running-config.

Command Mode

mlag Configuration

Command Syntax

local-interface vlan vlan_number

no local-interface

default local-interface

Parameters

vlan_number     VLAN number, in the range from 1 through 4094.

Guidelines

When configuring the local interface, the VLAN interface must exist already. To configure a VLAN interface, issue the command interface vlan.

Example

This command assigns VLAN 4094 as the local interface.

switch(config)# mlag
switch(config-mlag)# local-interface vlan 4094
switch(config-mlag)#

mlag (port-channel interface configuration)

The mlag command assigns an mlag ID to a port-channel. mlag peer switches form an mlag when each switch configures the same mlag ID to a port-channel interface. Only one mlag ID can be assigned to an interface. An individual mlag number cannot be assigned to more than one interface.

The no mlag and default mlag commands remove the mlag ID assignment from the configuration mode interface by deleting the corresponding mlag command from running-config.

Command Mode

Interface-Port Channel Configuration

Command Syntax

mlag number

no mlag

default mlag

Parameters

number     Number used as mlag ID. Value ranges from 1 to 2000.

Example

These commands configures a port channel and assigns it mlag 4.

switch(config)# interface ethernet 5-10
switch(config-if-Et5-10)# channel-group 1 mode active
switch(config-if-Et5-10)# interface port-channel 4
switch(config-if-Po4)# switchport trunk group group4
switch(config-if-Po4)# mlag 4
switch(config-if-Po4)# exit
switch(config)#

peer-address

The peer-address command specifies the peer IPv4 address for a Multi-chassis Link AGgregation (mlag) domain. mlag control traffic, including keepalive messages, is sent to the peer IPv4 address. If the peer IPv4 address is unreachable, then mlag peering fails and both peer switches revert to their independent state.

The no peer-address and default peer-address commands remove the mlag peer’s IPv4 address assignment by deleting the peer-address command from running-config.

Command Mode

mlag Configuration

Command Syntax

peer-address ipv4_addr

no peer-address

default peer-address

Parameters

ipv4_addr     mlag peer IPv4 address.

Example

These commands configure the mlag peer address.

switch(config)# mlag
switch(config-mlag)# peer-address 10.0.0.2
switch(config-mlag)#

peer-address heartbeat

The peer-address heartbeat command causes the mlag agent to start using Peer-IP address in the given VRF for UDP-based heartbeat control messages.

To enable mlag dual primary detection feature, the command must be configured on both mlag peers in the mlag config mode.

Command Mode

mlag configuration mode

Command Syntex

peer-address heartbeat Peer-IP [vrf vrf_name]

no peer-address heartbeat Peer-IP [vrf vrf_name]

default peer-address heartbeat Peer-IP [vrf vrf_name]

Parameters
  • Peer-IP The Management IP address of the mlag peer reachable in the VRF VRF-NAME (or default VRF if there is no VRF configured).
  • vrf vrf_nameNamed VRF.
Examples 
  • switch(config)# mlag
switch(config-mlag)# peer-address heartbeat 172.30.118.190
  • This example removes the feature. 
    switch(config-mlag)# no peer-address heartbeat

peer-link

The peer-link command specifies the interface that connects Multi-chassis Link AGgregation (mlag) peers. To form an mlag, two switches are connected through an interface called a peer link. The peer link carries control and data traffic between the two switches. Control traffic includes mlag-related advertisements and keepalive messages. This information keeps the two switches working as one.

The no peer-link and default peer-link command remove the peer link by deleting the peer-link command from running-config.

Command Mode

mlag Configuration

Command Syntax

peer-link INT_NAME

no peer-link

default peer-link

Parameters

INT_NAME denotes the interface type and number of the interface. Values include:

  • ethernet e_num     Ethernet interface range specified by e_num.

  • port-channel p_num     Channel group interface range specified by p_num.

Example

These commands creates a peer link.

switch(config)# mlag configuration
switch(config-mlag)# peer-link port-channel 10
switch(config-mlag)

mlag configuration (global configuration)

The mlag configuration command enters mlag configuration mode to configure Multi-chassis Link AGgregation (mlag) features. mlag configuration mode is not a group change mode; running-config is changed immediately after commands are executed. The exit command does not affect the configuration.

The no mlag configuration and default mlag configuration commands remove all mlag configuration commands from running-config.

The exit command returns the switch to global configuration mode.

Command Mode

Global Configuration

Command Syntax

mlag configuration

no mlag configuration

default mlag configuration

mlag and mlag configuration are identical commands.

Guidelines

An mlag is formed by connecting two switches through an interface called a peer link. The peer link carries control and data traffic between the switches, including advertisements and keepalive messages. This information coordinates the switches. Functioning peers are in the active state.

Each peer switch uses IP-level connectivity between their local addresses and the mlag peer IP address to form and maintain the peer link.

Commands Available in mlag Configuration Mode

Example

These commands enter mlag configuration mode and configure mlag parameters: 
switch(config)# mlag
switch(config-mlag)# local-interface vlan 4094
switch(config-mlag)# peer-address 10.0.0.2
switch(config-mlag)# peer-link port-channel 10
switch(config-mlag)# domain-id mlagDomain
switch(config-mlag)# heartbeat-interval 2500
switch(config-mlag)# reload-delay 2000
switch(config-mlag)# exit
switch(config)#

reload-delay mlag

The reload-delay mlag command configures the reload delay period for mlag links. The command also specifies the reload delay period for non-mlag links when the reload-delay non-mlag command is not configured.

Each Arista switch defaults to the recommended reload-delay value, which varies by switch platform:

  • Fixed configuration switches: 300 seconds
  • Trident II modular switches: 1200 seconds
    • 7304
    • 7308
    • 7316
    • 7300X series
  • Sand platform fixed configuration switches: 600 seconds
    • 7280 series (except 7280CR2 and 7280SR2)
    • 7020 series
  • Sand platform modular switches: 1800 seconds
    • 7504
    • 7508
    • 7500E series
    • 7548S
  • Sand Jericho+ fixed configuration switches: 900 seconds
    • 7280CR2 series
    • 7280SR2 series

The no reload-delay mlag and default reload-delay mlag commands restore the default value by deleting the reload-delay mlag statement from running-config.

Command Mode

mlag Configuration

Command Syntax

reload-delay [mlag] PERIOD

no reload-delay [mlag]

default reload-delay [mlag]

Parameters

  • PERIOD     Period that non-peer links are disabled after an mlag peer reboots. Options include:
    • infinity     link is not enabled after reboot.
    • 0 to 86400     disabled link interval (seconds). Default varies by switch platform as described above.

Guidelines

The reload-delay and reload-delay mlag commands are equivalent.

Example

These commands configure the reload-delay interval to 15 minutes.

switch(config)# mlag configuration
switch(config-mlag)# reload-delay mlag 900
switch(config-mlag)#

reload-delay mode

The reload-delay mode command specifies the state of LACP LAG ports during the mlag reload delay period. By default, mlag ports remain in the errdisabled state during reload delay. This command configures mlag ports to come up to standby mode before the expiration of the reload delay period.

The no reload-delay mode and default reload-delay mode commands restore the default behavior of mlag ports by deleting the reload-delay mode statement from running-config. The default behavior is for the mlag ports to remain in the errdisabled state until the expiration of the reload delay period

Command Mode

mlag Configuration

Command Syntax

reload-delay mode lacp standby

no reload-delay mode

default reload-delay mode

Related Commands

reload-delay mlag configures the mlag reload delay period.

Example

These commands configure the mlag port to come up to standby state before the end of the reload delay period.

switch(config)# mlag configuration
switch(config-mlag)# reload-delay mode lacp standby
switch(config-mlag)#

reload-delay non-mlag

The reload-delay non-mlag command specifies the period that non-mlag links are disabled after an mlag peer reboots. This interval allows non peer links to learn multicast and OSPF states before the ports start handling traffic. The recommended minimum value required to ensure the forwarding hardware is initialized with the topology state depends on the switch platform:
  • Fixed configuration switches: 300 seconds (five minutes)
  • Sand platform fixed configuration switches(7020 and 7280 series (except 7280CR2 and 7280SR2)): 600 seconds (ten minutes)
  • Modular switches: 1200 seconds (twenty minutes)

When the reload-delay non-mlag command is not configured, the reload-delay mlag command specifies the reload delay time for non-mlag and mlag links.

The no reload-delay non-mlag and default reload-delay non-mlag command restores the default behavior by deleting the reload-delay non-mlag statement from running-config.

Command Mode

mlag Configuration

Command Syntax

reload-delay non-mlag PERIOD

no reload-delay non-mlag

default reload-delay non-mlag

Parameters

PERIOD     Period that non-mlag links are disabled after an mlag peer reboots. Options include:
  • infinity     links are not enabled after reboot.
  • 0 to 86400     disabled link interval (seconds). Values range from 0 to 86400 (24 hours).

Example

These commands configure the reload-delay interval of non-mlag links to 20 minutes.

switch(config)# mlag configuration
switch(config-mlag)# reload-delay non-mlag 1200
switch(config-mlag)#

show mlag interfaces members

The show mlag interfaces members command displays information about the Multi-chassis Link AGgregation (mlag) members on bridged Ethernet interfaces.

Command Mode

EXEC

Command Syntax

show mlag interfaces members

Example

This command displays the mlag interface members.

switch# show mlag interface members
mlag4 is Port-Channel4
  Active Ports: Ethernet3 PeerEthernet3
mlag5 is Port-Channel5
  Active Ports: Ethernet14
mlag7 is Port-Channel7
  Active Ports: Ethernet5 PeerEthernet5
mlag8 is Port-Channel8
  Active Ports: Ethernet10 PeerEthernet10
mlag9 is Port-Channel9
  Active Ports: Ethernet15 Ethernet21 PeerEthernet19 PeerEthernet20
mlag10 is Port-Channel10
  Active Ports: Ethernet19 Ethernet20 PeerEthernet21 PeerEthernet22
switch#

show mlag interfaces states

The show mlag interfaces states command displays information about the Multi-chassis Link AGgregation (mlag) states on bridged Ethernet interfaces.

Command Mode

EXEC

Command Syntax

show mlag interfaces [mlagS] states [ STATE_NAMES][INFO_LEVEL]

Parameters
  • mlagS     mlag channels for which command displays data. Options include:
    • no parameter     command displays data for all mlags.
    • mlag_id     specifies mlag for which command displays data. Value ranges from 1 to 2000.
  • STATE_NAMES     mlag channels for which command displays data. Parameter may specify more than one name, which can be listed in any order. Valid state names include:
    • active-full     includes active-full interfaces.
    • active-partial     includes active-partial interfaces.
    • configured     includes configured interfaces.
    • disabled     includes disabled interfaces.
    • inactive     includes inactive interfaces.
  • INFO_LEVEL     specifies information displayed by command. Options include:
    • no parameter     command displays basic mlag interface parameters.
    • detail     command displays detailed mlag interface state parameters.

Example

This command displays the mlag interface states that are active-full.

switch# show mlag interfaces states active-full
                                                                   local/remote
  mlag     desc                       state     local     remote         status
-------- -------------------- --------------- --------- ---------- ------------
     4     b.po1                active-full       Po4        Po4          up/up
     7     ar.mg.au.po1         active-full       Po7        Po7          up/up
     8     co.po1               active-full       Po8        Po8          up/up
     9     k.po5                active-full       Po9        Po9          up/up
    10     ar.mg.pt.ir.po10     active-full      Po10       Po10          up/up
switch#

show mlag interfaces

The show mlag interfaces command displays information about the Multi-chassis Link AGgregation (mlag) configuration on bridged Ethernet interfaces.

Command Mode

EXEC

Command Syntax

show mlag interfaces [mlagS][INFO_LEVEL]

Parameters
  • mlagS     mlag channels for which command displays data. Options include:
    • no parameter     command displays data for all mlags.
    • mlag_id     specifies mlag for which command displays data. Value ranges from 1 to 2000.
  • INFO_LEVEL     specifies information displayed by command. Options include:
    • no parameter     command displays basic mlag interface parameters.
    • detail     command displays detailed mlag interface parameters.

Example

This command displays output from the show mlag interfaces detail command:

switch> show mlag interfaces detail
                                local/remote
mlag       state local remote   oper   config          last change  changes
----------------------------------------------------------------------------
   4 active-full   Po4    Po4  up/up  ena/ena  6 days, 1:19:26 ago        5
   5 active-full   Po5    Po5  up/up  ena/ena  6 days, 1:19:24 ago        5
   6 active-full   Po6    Po6  up/up  ena/ena  6 days, 1:19:23 ago        5
   7 active-full   Po7    Po7  up/up  ena/ena  6 days, 1:19:23 ago        5

show mlag issu warnings

The show mlag issu warnings command displays a warning message regarding the backward-compatibility of this feature before you upgrade.

Command Mode

EXEC

Command Syntax

show mlag issu warnings

Example

This command displays the mlag backward-compatibility warning message. Refer to the mlag ISSU Compatibility Matrix for details about upgrading the configuration.

switch# show mlag issu warnings
Please use the mlag ISSU Compatibility Check tool at https://www.arista.com/en/support/mlag-portal to verify ISSU compatibility.
If you are performing an upgrade, and the Release Notes for the new version of eos indicate that mlag is not backwards-compatible 
with the currently installed version (4.28.8.1M), the upgrade will result in packet loss.

Stp is not restartable. Topology changes will occur during the upgrade
process.

switch#

show mlag subinterfaces

The show mlag subinterfaces command displays information about mlag Layer 2 subinterfaces configured on a switch.

Command Mode

EXEC

Command Syntax

show mlag subinterfaces

Parameters

1_2048 - Specify the mlag interface ID.

Example

To display information about mlag L2 subinterfaces configured on a switch, use the show mlag subinterfaces command:

switch#show mlag subinterfaces
mlag       Desc        Local Sub-Interface    Local Status
---------- ---------- ---------------------  -------------
10.1                           Po20.1             up
10.2                           Po20.2             up
10.3                           Po20.3             up
11.1                           Po11.1             up

show mlag

The show mlag command displays information about the Multi-chassis Link AGgregation (mlag) configuration on bridged Ethernet interfaces.

Command Mode

EXEC

Command Syntax

show mlag [INFO_LEVEL]

Parameters

INFO_LEVEL     specifies information displayed by command. Options include:
  • no parameter     command displays mlag configuration, status, and ports.
  • detail     command displays mlag configuration, status, ports, and detailed status.

Example

This command displays output from the show mlag command:

switch> show mlag
mlag Configuration:
domain-id         :         ar.mg.mlag
local-interface   :           Vlan3901
peer-address      :       172.17.254.2
peer-link         :      Port-Channel1

mlag Status:
state             :             Active
peer-link status  :                 Up
local-int status  :                 Up
system-id         :  02:1c:73:00:13:19

mlag Ports:
Disabled          :   0
Configured        :   0
Inactive          :   0
Active-partial    :   0
Active-full       :   5
switch>

shutdown (mlag)

The shutdown command disables mlag on the switch without modifying the mlag configuration.

The no shutdown and default shutdown commands re-enable mlag by removing the shutdown command from running-config.

Command Mode

mlag Configuration

Command Syntax

shutdown

no shutdown

default shutdown

Example

These commands disable mlag on the switch.

switch(config)# mlag configuration
switch(config-mlag)# shutdown
switch(config-mlag)#