This Chapter provides step-by-step instructions for using the integrated Dell Remote Access Controller (iDRAC) Enterprise Version to install DMF software images on Dell server hardware.
This Chapter was validated using iDRAC Enterprise. The instructions are similar for the following installation options.
DMF Controller Release 6.3.1 on the Dell R440 Server
Analytics 6.3.1 on the Dell R440 Server
DMF Recorder on the Dell R740 Server
The instructions in this document tested using macOS are similar to those tested using Windows OS. You can use Internet Explorer, Chrome, or Safari browsers to access the iDRAC web interface.
Note: DMF Controller Nodes, Arista Analytics Nodes, DMF Recorder Nodes, and DMF Service Nodes using Dell R440/R640/R740 server hardware include an iDRAC Enterprise license.
Setting Up and Configuring iDRAC
To set up iDRAC on a Dell R440 or R740 server, complete the following steps.
Procedure
Turn on the Dell PowerEdge Server.
Connect a monitor to the VGA port and the keyboard to the USB port.
Note: When using the serial port to connect to the R440 and the R740, set the baud rate to 115200.
Press F2 to enter the System Setup Main Menu screen.
Figure 1. System Setup Main Menu
Select iDRAC Settings from the System Setup Main Menu (F2 > iDRAC Settings).
Figure 2. iDRAC Settings
Use the arrow keys to select Network.
Figure 3. iDRAC Settings Network
Select the Dedicated option from the NIC selection list for use with iDRAC. A dedicated IDRAC port is available on R440 and R740 Dell servers.
Configure the iDRAC IPv4 address, netmask, gateway, and DNS addresses.
After completing the iDRAC configuration, press the Esc key to display the Exit menu.
Select Save Changes and Exit and then press Enter to keep the changes.
From a web browser, type DRAC-IPv4-address in the browser address bar and press Enter.
The system displays the iDRAC web interface.
Figure 5. iDRAC Web Interface Login Window
Enter the username and password.
Default Password
By default, all PowerEdge servers ship with a unique iDRAC password, randomly generated at the factory, that is located on the pull-out Information Tag located on the front of the chassis, near the server asset tag. Customers who choose this option need to note this unique random password and use it to log in to iDRAC for the first time. For security purposes, Dell strongly recommends changing the default password.
Legacy Password
Customers who prefer to use the known legacy password calvin should choose this option. One reason to select this option would be to ensure conformance with any existing scripts. For security purposes, Dell strongly recommends changing the legacy password.
The pull-out information card is shown in the figure below:
Check out the bottom label on the pull-out information card to find the default password, which in this example is 94AXYXFERKNW:
Figure 6. Change the Default Password
To change the default password, type the new password, when prompted, and type it again to confirm.
In this example, the password stayed the same.
Log in to the iDRAC web interface using the new credentials (if changed).
Using the Virtual Console window option, click Launch Virtual Console.
Figure 7. System Summary Page
When prompted at the bottom of the screen, click Keep to confirm the operation.
Figure 8. System Summary Page
Click the viewer.jnlp link, as shown in the following example.
Figure 9. System Summary Page
When prompted, open the viewer.jnlp file.
Figure 10. System Summary Page
Note: Java is required, and the correct version downloads automatically.
Figure 11. System Summary: Continue Prompt
When prompted, click Continue.
Figure 12. System Summary: Run Prompt
When prompted, click Run.
Figure 13. Confirm Connection to Untrusted Network
When prompted to continue with an untrusted certificate, click Run.
Figure 14. System Summary: Confirmation Prompt
When prompted, click OK.
iDRAC launches the virtual console window.
Using iDRAC to Install a DMF Controller, Analytics, or Recorder Software Image
Before beginning, set up iDRAC as described in the Setting Up and Configuring iDRAC section, and then complete the instructions in this section. The procedure is similar for installing DMF Controller, Analytics, and Recorder software images.
After installing the software image using iDRAC, complete the installation and initial configuration described in this document's previous chapters.
To use iDRAC to install the software image, complete the following steps:
Direct your browser to the iDRAC web interface and log in to launch the iDRAC virtual console.
Figure 15. iDRAC Virtual Console Window
Select Virtual Media > Connect Virtual Media.
Figure 16. Virtual Media Connect Virtual Media Option
Click Virtual Media again and select Map CD/DVD this time.
Figure 17. Virtual Media Map CD/DVD Option
Click Browse to select the ISO file you want to install.
Figure 18. Virtual Media Map CD/DVD Browse Option
Select the Analytics, DMF Controller, or Recorder software image to install and click Open.
The following example shows selecting the Arista Analytics image.
Figure 19. Open DMF Controller ISO Image
Click Map Device.
Figure 20. Map Device
This action maps the DMF Controller ISO file to a Virtual CD/DVD on the Virtual Media menu.
Figure 21. Virtual Media DMF Controller ISO Mapped to a Virtual CD/DVD
Click Next Boot.
Figure 22. Next Boot Menu
Select Virtual CD/DVD from the Next Boot menu.
When prompted, click OK.
Figure 23. Next Boot Confirmation Prompt
Click Power and select Restart System (warm boot).
Figure 24. Power Restart System (warm boot)
When prompted, click OK.
Figure 25. Power Control Confirmation Prompt
When the server boots up, it selects the Virtual CD/DVD as the boot device.
Figure 26. Booting from Virtual CD/DVD
The server displays its status as it boots from Virtual CD/DVD.
Figure 27. Server Boot Status
Note: Depending on the network speed, it may take a while to download the ISO image to the server.
Figure 28. Server Boot Status
When prompted, type yes to install the DMF Controller image on the server.
Using iDRAC with a Dell R430, R630, or R730 Server
This chapter provides step-by-step instructions for using the integrated Dell Remote Access Controller (iDRAC) Enterprise Version to install the DMF Controller image on Dell R430 servers and the DMF Service Node Appliance image on Dell R630 or R730 servers.
This chapter was validated using iDRAC Enterprise Version 2.10.10 and DMF Release 6.0.1. The instructions were tested with DMF Controller Release 6.0.1 on Dell R430 and DMF Service Node Appliance Release 6.0.1 on Dell R630 or R730.
The instructions in this chapter tested using macOS are similar to those tested using Windows OS. Use Internet Explorer, Chrome, or Safari browsers to access the iDRAC web interface.
Note: The iDRAC features require a separate iDRAC Enterprise license on the Dell R430/R630/R730 hardware appliances.
To purchase an iDRAC Enterprise license, contact Dell Sales at https://www.dell.com/learn/us/en/19/campaigns/contact-us-phone-number.
Alternatively, purchase an iDRAC Enterprise license from Dell Digital Locker (DDL). https://www.dell.com/support/kbdoc/en-us/000130349/how-to-obtain-idrac-enterprise-licenses-from-dell-digital-locker-ddl.
Setting Up and Configuring iDRAC
Complete the following steps to set up iDRAC on a Dell R430, R630, or R730 server.
Procedure
Turn on the Dell PowerEdge Server.
Connect a monitor to the VGA port and the keyboard to the USB port.
Note: When using the serial port to connect to the R430, R630, or R730, set the baud rate to 115200.
Press <F2> to enter the System Setup Main Menu screen.
Figure 1. System Setup Main Menu
Select iDRAC Settings from the System Setup Main Menu (F2 > iDRAC Settings).
Figure 2. iDRAC Settings
Use the arrow keys to select Network.
Figure 3. iDRAC Settings > Network
Select an option from the NIC Selection list for use with iDRAC. In this example, LOM1.
The options provided by both Dell R430 and R630 or R730 servers include LOM1, LOM2, LOM3, and LOM4. In addition to the LOM options, the Dell R630 or R730 provides a dedicated NIC for use with iDRAC. To use the dedicated NIC with a Dell R630 or R730, select Dedicated from the NIC Selection list.
Configure the iDRAC IPv4 address, netmask, gateway, and DNS addresses, as shown on the following page.
After completing the iDRAC configuration, press the Esc key to display the Exit menu.
Select Save Changes and Exit and then press Enter to keep the changes.
From a web browser, type DRAC-IPv4-address in the browser address bar and press Enter.
The system displays the iDRAC web interface.
Figure 5. iDRAC Web Interface Login Window
Enter the username and password.
Default Password
By default, all PowerEdge servers ship with a unique iDRAC password, randomly generated at the factory, that is located on the pull-out Information Tag located on the front of the chassis, near the server asset tag. Customers who choose this option need to note this unique random password and use it to log in to iDRAC for the first time. For security purposes, Dell strongly recommends changing the default password.
Legacy Password
Customers who prefer to use the known legacy password calvin should choose this option. One reason to select this option would be to ensure conformance with any existing scripts. For security purposes, Dell strongly recommends changing the legacy password.
The pull-out information card is shown in the figure below:
Check out the bottom label on the pull-out information card to find the default password, which in this example is 94AXYXFERKNW:
Figure 6. Change the Default Password
To change the default password, type the new password, when prompted, and type it again to confirm. In this example, the password stayed the same.
Figure 7. Licensing Page
To apply the iDRAC Enterprise Version license select the license option and then select import.
Select the license file and click Apply.
After applying the Enterprise Version license, log out for the license to take effect.
Note: You need to log out from the iDRAC web interface, but there is no need to close the web browser.
Log in to the iDRAC web interface using the new credentials (if changed).
After applying the license, logging out and logging in enables the Enterprise Version license.
Using the Virtual Console Preview option, click Launch.
Figure 8. System Summary Page
When prompted, click Keep to confirm the operation.
Figure 9. System Summary Page
Click the viewer.jnlp link, as shown in the following example.
Figure 10. System Summary Page
When prompted, open the viewer.jnlp file.
Figure 11. System Summary Page
Note: JAVA is required, and the correct version downloads automatically.
Figure 12. System Summary: Continue Prompt
When prompted, click Continue.
Figure 13. System Summary: Run Prompt
When prompted, click Run.
Figure 14. Confirm Connection to Untrusted Network
When prompted to continue with an untrusted certificate, click Run.
Figure 15. System Summary: Confirmation Prompt
When prompted, click OK.
iDRAC launches the virtual console window.
Using iDRAC to Install the DMF Controller or DMF Service Node Image
Before beginning, set up iDRAC as described in the Setting Up and Configuring iDRAC section, and then complete the instructions in this section.
After installing the Controller software image using iDRAC, follow the instructions in Installing and Configuring the DMF Controller to complete the DMF Controller installation and initial configuration.
For DMF Release 6.3.2 onwards, the procedure for using iDRAC to install the DMF Controller or DMF Service node image is the same.
Complete the following steps using iDRAC to install the DMF Controller or Service Node image on a Dell R630/R730.
Procedure
Direct your browser to the iDRAC web interface and log in to launch the iDRAC virtual console.
Figure 16. iDRAC Virtual Console Window
Select Virtual Media > Connect Virtual Media.
Figure 17. Virtual Media > Connect Virtual Media Option
Click Virtual Media again and select Map CD/DVD this time.
Figure 18. Virtual Media > Map CD/DVD Option
Click Browse to choose the DMF Controller ISO file.
Figure 19. Virtual Media > Map CD/DVD Browse Option
Select the DMF Controller ISO image and click Open.
Figure 20. Open DMF Controller ISO Image
Click Map Device.
Figure 21. Map Device
This action maps the DMF Controller ISO file to a Virtual CD/DVD on the Virtual Media menu.
Figure 22. Virtual Media > DMF Controller ISO Mapped to a Virtual CD/DVD
Click Next Boot.
Figure 23. Next Boot Menu
Select Virtual CD/DVD from the Next Boot menu.
Figure 24. Next Boot > Virtual CD/DVD/ISO Option
When prompted, click OK.
Figure 25. Next Boot Confirmation Prompt
Click Power and select Restart System (warm boot).
Figure 26. Power Restart System (warm boot)
When prompted, click OK.
Figure 27. Power Control Confirmation Prompt
When the server boots up, it selects the Virtual CD/DVD as the boot device.
Figure 28. Booting from Virtual CD/DVD
The server displays its status as it boots from Virtual CD/DVD.
Figure 29. Server Boot Status
Note: Depending on the network speed, it may take a while to download the ISO image to the server.
Figure 30. Server Boot Status
When prompted, type yes to install the DMF Controller image on the server.
Figure 31. Installation Prompt
To complete the installation and initial configuration, refer to the instructions in this document.
This chapter describes options for increasing the security of management access to the DMF Controller node.
Management Plane Security
The management plane network is used by the administrator, whether locally or remotely, to reach the Controller management interfaces. DANZ Monitoring Fabric (DMF) uses standard, well-known cryptographic technology, such as RSA and AES. Still, system administrators must choose strong passwords and change them frequently, according to well-established security best practices.
All services the Controller uses are enabled by default except for SNMP, which is disabled by default. Refer to the Protocol Access Required to the DMF Controller section to block or permit specific protocols to the management interface.
For example, the control plane is the network between the Controllers and the switches to carry OpenFlow control traffic. The following are general requirements and recommendations for deployment:
The Controller must be on the same Layer 2 network as the switches—physically isolated data, control, and management plane networks.
The only devices on the control plane are switches and Controllers.
Make the control plane network not routed or minimally IP access restricted via its egress router.
Physically secure the management and data plane networks (for example, locks on the cage doors).
Many of the Zero-Touch Networking (ZTF) protocols (DHCP, such as ONIE, Controller discovery, and image download) and the OpenFlow protocol are not authenticated. They are subject to spoofing in an untrusted network. The following are best practices regarding securing the control plane within the switched fabric.
The control plane network is “Layer 2 trusted,” meaning the attacker cannot spoof Layer 2 messages on the control network. In practice, this means the control plane network should be an isolated VLAN, ideally containing only the Controller and switches.
Harden the switch management interface against Layer 3 attacks (all services are authenticated, unnecessary services are turned off, and so forth).
The network should not be reachable by Layer 3 protocols. If Layer 3 access is required, the administrator should maintain a Layer 3 allowlist of hosts that can access the control network, for example, using an ACL on the edge router.
Importing the Controller Private Key and Certificate
This section describes how to import a private key and a certificate to the Controller after copying it to the Controller using the copy command.
To import a private key to the Controller, enter the private-key command in the config-controller submode:
[no] private-key <controller-key-name>
Replace controller-key-name with the name of the private key. Use the no version of the command to remove the private-key.
To import the Controller certificate, use the certificate command in config-controller submode.
[no] certificate <name>
Replace the name with the name assigned to the Controller certificate. Use the no version of the command to remove the certificate.
Import the private key and certificate to the Controller using the copy command.
Using Certificates Signed by a CA for GUI Access to the Controller
By default, SSL is enabled on the Controller using a self-signed certificate. Complete the following steps to install a certificate signed by a public or private CA.
Procedure
Generate the Certificate Signing Request (CSR) and the private key for the Controller.
Perform this operation on any workstation that supports OpenSSL. The following example shows the operation performed on a Linux workstation.
root@Ubuntu-12:~/openssl-ca/admin# openssl req -newkey rsa:2048 -nodes -keyout controller.
key -new -out controller.csr
Generating a 2048 bit RSA private key
.......................................+++
...............................+++
writing new private key to 'controller.key'
-----
You are about to be asked to enter information that will be incorporated into your
certificate request.
What you are about to enter is what is called a Distinguished Name or a DN. There are
quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:California
Locality Name (eg, city) []:Santa Clara
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Arista Networks Organizational
Unit Name (eg, section) []:Engineering
Common Name (e.g. server FQDN or YOUR name) []:DMF Secure Certificate
Email Address []:This email address is being protected from spambots. You need JavaScript enabled to view it.
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:anet1234
An optional company name []:Arista
root@Ubuntu-12:~/openssl-ca/admin#
root@Ubuntu-12:~/openssl-ca/admin# ls -ltr
total 8
-rw-r--r-- 1 root root 1708 Feb 7 15:39 controller.key
-rw-r--r-- 1 root root 1184 Feb 7 15:39 controller.csr
root@Ubuntu-12:~/openssl-ca/admin#
Submit the CSR to the CA and get the certificate signed.
Submit the CSR to the trusted CA for browsers used to access the DMF GUI. For organizations using GUI-based CAs, copy the contents of the CSR to the CA for signature.
The following example shows the operation performed on a Linux workstation.
root@Ubuntu-12:~/openssl-ca# openssl ca -config openssl-ca.cnf -policy signing_policy -
extensions signing_req -out admin/controller.pem -infiles admin/controller.csr
Using configuration from openssl-ca.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
countryName :Printable:'US'
stateOrProvinceName :ASN.1 12:'California'
localityName :ASN.1 12:'Santa Clara'
organizationName :ASN.1 12:'Arista Networks'
organizationalUnitName:ASN.1 12:'Engineering'
commonName :ASN.1 12:'DMF Secure Certificate'
Certificate is to be certified until Nov 3 23:41:17 2020 GMT (1000 days) Sign the
certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new
entries
Data Base Updated
root@Ubuntu-12:~/openssl-ca#
root@Ubuntu-12:~/openssl-ca/admin# ls -ltr
total 16
-rw-r--r-- 1 root root 1708 Feb 7 15:39 controller.key
-rw-r--r-- 1 root root 1184 Feb 7 15:39 controller.csr
-rw-r--r-- 1 root root 5882 Feb 7 15:41 controller.pem
root@Ubuntu-12:~/openssl-ca/admin#
Copy the signed certificate to the Controller:
controller-1# copy scp://This email address is being protected from spambots. You need JavaScript enabled to view it..3:/root/openssl-ca/admin/controller.pem cert://
This email address is being protected from spambots. You need JavaScript enabled to view it..3's password:
controller.pem
5.74KB - 00:00
controller-1# copy scp://This email address is being protected from spambots. You need JavaScript enabled to view it..3:/root/openssl-ca/admin/controller.key private-key:/
/controller- private.key
This email address is being protected from spambots. You need JavaScript enabled to view it..3's password:
controller.key
1.67KB - 00:00
controller-1#
controller-1(config-controller)# show this
! controller
controller
certificate 'DMF Secure Certificate'
cluster-name DMF_Cluster
private-key controller-private.key
access-control
!
access-list api
1 permit from ::/0
2 permit from 0.0.0.0/0
!
access-list gui
1 permit from ::/0
2 permit from 0.0.0.0/0
!
access-list ssh
1 permit from ::/0
2 permit from 0.0.0.0/0
controller-1(config-controller)#
Access the DMF GUI using a browser and display the certificate.
After connecting to the Controller, click the padlock icon to the left of the location field to display information about the certificate.
Replacing the Certificate
Please use the following steps to replace the Controller's certificate.
Scenario 1: Using the same CSR used to sign the current certificate.
Obtain a newly signed certificate from CA using the same CSR and copy it to the Controller using the following command:
# copy new certificate from the source cert://
For example:
# copy scp://This email address is being protected from spambots. You need JavaScript enabled to view it./root/openssl-ca/certificate.pem cert:// This email address is being protected from spambots. You need JavaScript enabled to view it.password certificate.pem
6.49KB - 00:00
#
No other action is needed as the current certificate will be overwritten when copying the new one.
Scenario 2: Does not have the same CSR for the current certificate.
Generate a new CSR and the private key.
Sign the CSR to get the new certificate.
Import/copy the certificate to the Controller. The current certificate will be overwritten if the Common Name matches the new one.
Import/copy the new private key to the Controller. The private key will be overwritten if the file name is the same as the old one. In that case, there is no need for any config changes.
Assuming the Common Name and the private key dest file names are different from the original ones, remove the old certificate and private key and install a new certificate and private key.
To remove the old certificate and private key, use the following commands:
C1(config)# controller
C1(config-controller)# no certificate certificate name
C1(config-controller)# no private-key private-key name
C1(config-controller)#
To configure the new certificate and private key, use the following commands:
C1(config)# controller
C1(config-controller)# certificate new certificate name
C1(config-controller)# private-key new private-key name
C1(config-controller)#
Managing the Controller HTTP and SSH Ciphers, Protocols, and Data Integrity Algorithms
Use the crypto command to enter the config-crypto submode to configure settings for HTTP and SSH. Use the http and ssh commands in the config-crypto submode to configure the ciphers and protocols. Configure the list of enabled ciphers, protocols, or algorithms by appending to the list.
Use the no version of this command with any keyword to remove the specific cipher, protocol, or algorithm. Use the no version of the command without a keyword to restore the list to the default value. Use the CLI help feature to identify the supported ciphers, protocols, or data integrity (MAC) algorithms.
Configuring HTTP Ciphers
Enter the following commands to configure the ciphers for HTTP:
Note: When you configure a set of ciphers instead of using the default set, please make sure to configure at least one or all the three ciphers mentioned below in addition to your choice. ECDHE-RSA-CHACHA20-POLY1305, ECDHE-RSA-AES128-GCM-SHA256, ECDHE-RSA-AES256-GCM-SHA384.
Starting in the DANZ Monitoring Fabric 8.4 release, the TLSv1.3 HTTPS protocol is supported. DMF supports TLSv1.3 and TLSv1.2 by default, with the TLSv1.3 protocol preferred for TLS connections.
Enter the following commands to configure the protocols for HTTP:
Configured SSH ciphers and MAC algorithms on the Controller are pushed to the switches running Switch Light OS via ZTN. With this enhancement, users can also restrict the SSH ciphers and MAC algorithms on the switches.
Enter the following commands to configure the ciphers for SSH:
Enter the following command to configure data integrity (MAC) algorithms for SSH:
controller-1(config)# crypto
controller-1(config-crypto)# ssh
controller-1(config-crypto-ssh)# mac <index><mac-name>
Changes to Supported MACs/Ciphers/SSH Keys
Please note the below changes to supported MACs/Ciphers and SSH Keys in DANZ Monitoring Fabric starting with the DMF 8.0 release due to the upgrade of Ubuntu OS from 16.04 to 20.04. Arista Networks refined the list of supported ciphers to represent better what can be configured and those default enabled when TLSv1.3 protocol is enabled.
This email address is being protected from spambots. You need JavaScript enabled to view it.This email address is being protected from spambots. You need JavaScript enabled to view it.This email address is being protected from spambots. You need JavaScript enabled to view it.This email address is being protected from spambots. You need JavaScript enabled to view it.
hmac-sha2-512
hmac-sha2-256
This email address is being protected from spambots. You need JavaScript enabled to view it.
hmac-sha1
The following SSH MACs are obsolete and no longer supported:
hmac-ripemd160
This email address is being protected from spambots. You need JavaScript enabled to view it.This email address is being protected from spambots. You need JavaScript enabled to view it.
The following SSH ciphers are obsolete and no longer supported:
ssh_host_dsa_key is obsolete and no longer supported.
ssh_host_ed25519_key is now available, along with ssh_host_ecdsa_key
and ssh_host_rsa_key that have been available since past releases.
This feature provides the ability to mirror the SSH/HTTPS cryptographic configuration of the DMF Controller to the managed appliances (i.e., service nodes and recorder nodes) and the SSH cryptographic configuration of the Controller to the EOS switches.
Using the CLI to Configure SSH and HTTPS
The configuration that a managed appliance or EOS switch receives is intended for the Controller itself. Configuring a cipher or message authentication code (MAC) on the Controller will automatically be reflected onto a managed appliance or EOS switch.
Check the cryptographic configuration of the Controller using the show
running-config command, as shown in the example below, and verify the settings in the crypto section.
# show running-config
.
.
.
! crypto
crypto
!
ssh
cipher 1 3des-cbc
mac 1 hmac-md5
.
.
.
All ciphers/protocols/MACs of the HTTPS/SSH cryptographic configuration supported on the Controller are supported on the managed appliances, with one caveat listed in the Limitations section below. Check the HTTPS/SSH cryptographic configuration by reviewing the running-config of a managed appliance, as shown below for a Recorder Node.
Note: EOS does not support all SSH ciphers and MACs that the Controller does.
The following SSH MAC algorithms supported by the Controller are not supported by EOS:
This email address is being protected from spambots. You need JavaScript enabled to view it. (HMAC-MD5 in “encrypt-then-mac” mode)
This email address is being protected from spambots. You need JavaScript enabled to view it. (HMAC-MD5 in “encrypt-then-mac” mode)
This email address is being protected from spambots. You need JavaScript enabled to view it. (HMAC-SHA1 in “encrypt-then-mac” mode)
This email address is being protected from spambots. You need JavaScript enabled to view it. (message authentication code based on universal hashing (UMAC) in “encrypt-then-mac” mode)
This email address is being protected from spambots. You need JavaScript enabled to view it. (UMAC)
The following SSH cipher algorithm supported by the Controller is not supported by EOS:
This email address is being protected from spambots. You need JavaScript enabled to view it. (Rijndael in CBC mode)
This difference can be seen when reviewing the running-config of the Controller and the ZTN-generated running-config of an EOS switch:
# show running-config
.
.
.
.
! crypto
crypto
!
ssh
cipher 1 This email address is being protected from spambots. You need JavaScript enabled to view it.
cipher 2 3des-cbc
mac 1 This email address is being protected from spambots. You need JavaScript enabled to view it.
mac 2 This email address is being protected from spambots. You need JavaScript enabled to view it.
.
.
.
.
# show switch switch-name running-config
.
.
.
cipher 3des-cbc
mac This email address is being protected from spambots. You need JavaScript enabled to view it.
.
.
.
Only the ciphers/MACs that are supported get added to the running-config of the EOS switch. To review the disallowed MACs/ciphers when generating the running-config of the switch, use the following show command:
# show fabric warnings feature-unsupported-on-device
# NameWarning
-|-----|---------------------------------------------------------------------|
1 core1 This email address is being protected from spambots. You need JavaScript enabled to view it. is not a supported cipher on EOS switches
2 core1 This email address is being protected from spambots. You need JavaScript enabled to view it. is not a supported mac on EOS switches
Syslog Messages
No syslog messages are generated when the DMF Controller’s cryptographic configuration is mirrored to the managed appliances and EOS switches.
Limitations
There are limitations to the HTTPS configuration (some options may cause ZTN protocol/communication and controller-to-controller communication failures). The following HTTPS protocol versions are required to be used to avoid communication failures:
TLSv1.2
TLSv1.3
It is not apparent when a cipher/MAC is not reflected onto an EOS switch (due to it being unsupported). To ascertain this scenario,check the Controller's running-config and the switch's ZTN-generated running-config and compare them (alternatively, check the “show fabric warnings” command output to review any generated warnings).
An ECDSA-based cryptographic cipher configuration inherited by the managed appliances will cause a failure in communication with the Controller.
Protocol Access Required to the DMF Controller
Management Plane Access
You can control access to the DMF Controller for specific protocols, and in the case of SSH, you can allow access only from specific IP addresses or subnetworks. The following table summarizes the TCP/UDP protocol ports that DMF uses. The CLI access-list option column shows the options for the ports that can be enabled or disabled using the CLI access-list command (config-controller-access submode). The ports listed are open by default on the Controller, except for SNMP, which is disabled by default.
These ports must also be open on any device, such as a router or firewall, that connects the management console or application to the DMF Controller.
Protocol
Port
Application
CLI access-list option
Match criteria
HTTP
TCP 80
GUI auto-redirect
HTTPS
TCP 443
GUI remote access
gui, applicable to Controller, Service Node, Recorder Node, Analytics Node
Default any, configurable
HTTPS
TCP 8443
REST API
api, applicable to Controller, Service Node, Recorder Node, Analytics Node
Default any, configurable
ICMP/ICMPv6
ICMP/ICMPv6
selected ICMP types
ICMP/ICMPv6
ICMP/ICMPv6
selected ICMP types
SNMP
UDP 161, 162
SNMP, applicable to Controller, Service Node, Analytics Node
snmp
Default none, configurable
SSH
TCP 22
CLI remote access
ssh, applicable to Controller, Service Node, Recorder Node, Analytics Node
Default any, configurable
syslog
UDP 514
vce-api
UDP 7443
vCenter integration
vce-api
Enabled by default
Note: Be careful when configuring firewall rules for the SSH protocol, which is permitted from all subnetworks by default. The option exists to restrict SSH access to one or more specific subnetworks. However, this denies access from all other subnetworks. If no connectivity from a specified subnetwork is available, accessing the Controller is only through the local console.
Control Plane Access for DMF Controller
The following ports must be open between the DMF Controller and any connected devices. No further configuration is required if all devices are in the same Layer 2 network as the DMF Controller. However, if any DMF nodes or fabric switches connect over a Layer 3 network, these ports must be open on any firewalls or routers that connect the devices to the DMF Controller.
Table 1. DMF Controller
Protocol
Port
Direction
Application
In Flows
Out Flows
TCP
22
Both Directions
SSH
Customer
Switches, managed appliances
TCP
49
Out
TACACS+
Customer
TACACS+ server
TCP
53
Out
DNS
Customer DNS server
UDP
53
Out
DNS
Customer DNS server
UDP
67
Out
DHCP
Customer DHCP server
UDP
68
In
DHCP
Customer DHCP server
TCP
80
In
ZTN ONIE
Switches
UDP
123
Both Directions
NTP
Switches, Service Node, Recorder Node, Analytics Node
Customer NTP server
UDP
161
In
SNMP
Customer
UDP
162
Out
SNMP Traps
Customer
TCP
443
In
GUI
Customer
UDP
514
Out
Syslog
Customer Syslog server
UDP
1813
Out
RADIUS
Default RADIUS accounting port
UDP
5353
In
ZTN MDNS
Switches, Service Node, Recorder Node
TCP
6379
Out
Controller Stats
Analytics Nodes
TCP
6642
Both Directions
Cluster Sync
Controller HA
Controller HA
TCP
6653
In
OpenFlow
Switches, Recorder Node, Service Node
TCP
7443
In
VCE API
vCenter API
TCP
8443
Both Directions
Floodlight
REST API
Customer, Recorder Node
Recorder Node, Service Node
TCP
8443
Out
Controller-to-switch traffic
Fabric switches using the Switch Light OS
TCP
8843
In
ZTN
Switches, Service Node, Recorder Node
TCP
9379
Out
Analytics Node Replicated Redis
Analytics Node
Note:Starting from DMF release 8.5.0, Controllers use port 8443 to apply configurations to fabric switches using the Switch Light OS. In earlier releases it was port 443. If using Switch Light OS, network administrators must update the firewall configuration to permit traffic on port 8443 between Controllers and fabric switches.
To enable SNMP access to the Controller or to restrict access to the Controller for the REST API, web-based GUI access, or SSH applications, complete the following steps.
Procedure
Enter the controller command from config mode to enter config-controller submode.
Enter the access-list command from config-controller-access submode followed by the protocol for which you want to configure a rule.
The protocols for which you can configure rules include the following:
api: Enter the config-controller-access-list submode for REST/API access to the Controller.
gui: Enter the config-controller-access-list submode for web-based GUI access to the Controller.
ns-api: Enter the config-controller-access-list submode to manage NS-API access to the Controller, for example, from OpenStack or vCenter.
ssh: Enter the config-controller-access-list submode for SSH access to the Controller.
snmp: Enter the config-controller-access-list submode for SNMP access to the Controller.
By default, the access list for all services except for SNMP is 0.0.0.0/0, which allows access from any IPv4 subnetwork and ::/0, which allows access from any IPv6 subnetwork. For SNMP, the access list is empty, meaning access is not permitted unless specifically enabled. SNMP UDP port 161 is blocked on Controllers and fabric switches by default. You must configure an SNMP access list using the Controller's CLI to communicate using SNMP.
For example, the following command enters config-controller-access-list submode for the SSH protocol:
When configuring the following access list on the Controller, the ACL is pushed to all connected switches, and the SNMP ACL is applied to each switch ma1 management interface.
controller-1(config-controller-access)# access-list snmp
controller-1(config-controller-access-list)# 1 permit from ::/0
controller-1(config-controller-access-list)# 2 permit from 10.0.0.0/8
Specify the subnetworks from which access is permitted for the specified protocol. Specify the subnetwork followed by a slash and the number of bits in the subnet mask.
For example, the following commands allow access to the SSH protocol only from the subnetwork 192.168.1.0:
controller-1(config-controller-access)# access-list
ssh controller-1(config-controller-access-list)# 10 permit from 192.168.1.0/24
To view the current firewall configuration for the Controller, enter the show
running-config command and see the access-control section.
Protocol Access Required to the DMF Controller - Sync
Sync has been added to the access list. All traffic is permitted for IPv4 (0.0.0.0/0) and IPv6 (::/0) by default. If the active standby Controller High Availability (HA) pair is deployed in different L3 subnets, permitting all traffic for sync services can be a security risk. For a secure connection between the active and standby Controller, the access list for syncing should only permit the active and standby Controller's management IP addresses.
Procedure
Add a rule to permit active Controller management IP address. Do not overwrite the existing default rule for sync. Use a different rule number when adding a new rule for the sync access list. The example below shows how to configure Rule ID 3 (3 permit from
10.240.130.17/32).
Add another rule to permit standby Controller management IP address. Do not overwrite the existing default rule for sync. Use a different rule number when adding a new rule for sync access list. The example below shows how to configure Rule ID 4 (4 permit from
10.240.130.16/32).
Verify cluster state using show controller details and make sure cluster state is redundant.
DMF-CTL2(config)# show controller details
Cluster Name : DMF-7050
Cluster UID : a5de38214971de42aa7b51b96ac7345f4f228b20
Cluster Virtual IP : 10.240.130.18
Redundancy Status : redundant
Redundancy Description : Cluster is Redundant
Last Role Change Time : 2022-11-05 00:56:04.862000 UTC
Cluster Uptime : 2 months, 1 week
# IPHostname @ Node Id Domain Id State StatusUptime
-|-------------|--------|-|-------|---------|-------|---------|---------------|
1 10.240.130.17 DMF-CTL2 * 22049 1 activeconnected 2 weeks, 2 days
2 10.240.130.16 DMF-CTL1 27671 1 standby connected 2 weeks, 2 days
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Failover History ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# New Active Time completed NodeReasonDescription
-|----------|------------------------------|-----|---------------------|-------------------------------------------------------|
1 220492022-11-05 00:55:35.994000 UTC 22049 cluster-config-change Changed connection state: cluster configuration changed
DMF-CTL2(config)#
Removing default rules before adding the new sync access-list rules can break Controller cluster communication. To recover from this, please refer to Controller Cluster Recovery.
Control Plane Access for DMF Switches
The following ports must be open for DMF Switches to communicate with DMF Controller, Analytics Node, and other services (e.g., NTP, DHCP, etc.). No further configuration is required if all devices are in the same Layer 2 network as the DMF Controller. However, if any DMF Controller and fabric switches connect over a Layer 3 network, these ports must be open on any firewalls or routers that connect the devices to the DMF Controller.
The following ports must be open for the DMF Service Node to communicate with the DMF Controller, Analytics Node, and other services (e.g., NTP, DHCP, etc.). No further configuration is required if all devices are in the same Layer 2 network as the DMF Controller. However, if the DMF Controller and Service Nodes connect over a Layer 3 network, these ports must be open on any firewalls or routers.
Table 3. DMF Service Node
Protocol
Port
Direction
Application
In Flows
Out Flows
TCP
22
In
SSH
Customer, DMF Controller
TCP
49
Out
TACACS+
Customer TACACS+ Server
TCP/UDP
53
Out
DNS
Customer DNS Server
UDP
67
Out
DHCP
Customer DHCP Server
UDP
68
In
DHCP
Customer DHCP
UDP
123
Out
NTP
Customer NTP Server
UDP
161
In
SNMP
Customer
UDP
162
Out
SNMP Trap
Customer SNMP Trap Server
UDP
514
Out
Syslog
Customer Syslog Server
UDP
1812
Out
Default RADIUS
Authentication port
Customer RADIUS Server
UDP
1813
Out
Default RADIUS
Accounting port
Customer RADIUS Server
UDP
5353
Out
ZTN MDNS
DMF Controller
TCP
6653
Out
OpenFlow
DMF Controller
TCP
8443
Both Direction
Floodlight REST
API
DMF Controller
DMF Controller
TCP
8843
Out
ZTN
DMF Controller
Control Plane Access for DMF Recorder Node
The following ports must be open between the DMF Recorder Node and any connected devices. No further configuration is required if all devices are in the same Layer 2 network as the DMF Recorder Node. However, if the DMF Controller, Analytics Node, or fabric switches connect over a Layer 3 network, these ports must be open on any firewalls or routers that connect the devices to the DMF Recorder Node.
Table 4. DMF Recorder Node
Protocol
Port
Direction
Application
In Flows
Out Flows
TCP
22
In
SSH
Customer, DMF Controller
TCP
49
Out
TACACS+
Customer TACACS+ Server
TCP/UDP
53
Out
DNS
Customer DNS Server
UDP
67
Out
DHCP
Customer DHCP Server
UDP
68
In
DHCP
Customer DHCP Server
UDP
123
Out
NTP
Customer NTP Server
UDP
161
In
SNMP
Customer
UDP
162
Out
SNMP Trap
Customer SNMP Trap Server
TCP
443
In
Stenographer
Query API
Customer, DMF Controller
UDP
514
Out
Syslog
Customer Syslog Server
UDP
1812
Out
Default RADIUS
Authentication port
Customer RADIUS Server
UDP
1813
Out
Default RADIUS
Accounting port
Customer RADIUS Server
TCP
2049
Both
NFS
Customer NSF Server
UDP
2049
Both
NFS
Customer NFS Server
UDP
5353
Out
ZTN MDNS
DMF Controller
TCP
6653
Out
OpenFlow
DMF Controller
TCP
8443
Both Direction
Floodlight REST
API
DMF Controller
DMF Controller
TCP
8843
Out
ZTN
DMF Controller
Control Plane Access for Analytics Node
The following ports must be open between the Analytics Node and any connected devices. No further configuration is required if all devices are in the same Layer 2 network as the Analytics Node. However, if the Analytics Node connects over a Layer 3 network, these ports must be open on any firewall or router.
Using Authentication, Authorization, and Accounting
This chapter describes how to manage administrative access to the DANZ Monitoring Fabric (DMF) Controller using local groups and users (RBAC) and AAA servers in general, and it also provides specific configuration for TACACS+ or RADIUS servers.
Overview
Access privileges to the DANZ Monitoring Fabric (DMF) Controller are associated with groups. Each user assigned to the group obtains the access permissions associated with the group. The current version of DMF supports the following two groups:
Admin Group: Root privileges with access to all modes, including debug modes.
Read-Only: Read-only administrative access.
DMF also supports communication with a remote AAA server. All authentication, authorization, and accounting functions are set to local by default. The general options that control where these functions occur are:
Local only: The accounts on the remote AAA server are ignored, using only the local database accounts.
Local primary and remote backup: The accounts on the local database are used first. If the account is not found locally, DMF uses the database on the remote server.
Remote only: Authentication and authorization do not fall back to local for users other than the admin and recovery accounts, no matter what happens.
Remote primary and local backup: Uses the accounts on the remote AAA database first. If the remote server is unavailable, DMF uses the local database accounts.
Note: The admin and recovery user accounts are special accounts that cannot be authenticated remotely using RADIUS or TACACS+. These accounts are always authenticated locally to prevent administrative access from being lost in case a remote AAA server is unavailable. The following list summarizes the options available for each function:
The following list summarizes the options available for each function:
Accounting: local, local and remote, or remote.
Authentication: local, local then remote, remote then local, or remote.
Authorization: local, local then remote, remote then local, or remote.
Note: Authorization falls back to local only if remote authorization fails because the remote AAA server is unreachable.
Using Local Groups and Users to Manage DMF Controller Access
Administrative access to the monitoring fabric is managed by assigning interfaces to groups and then assigning user accounts to the group, which can then view and use the assigned interfaces.
Viewing Existing Groups and Users
To view the current allocation and assignment of groups and resources, enter the following command from any CLI mode on the active DANZ Monitoring Fabric (DMF) Controller.
To use the GUI to manage groups and users, select Security from the DMF GUI main menu.
Figure 1. Security
The system displays the Security page menu options.
Users
Groups
Sessions
Login/Lockout History
HTTP/SSH Ciphers
Groups
Displays the following preconfigured default groups on the DMF Controller:
admin group: Provides full administrative access to all interfaces. Global configuration options, such as assigning a role to a switch or interface, must be performed by the admin user.
read-only group: Provides access to view and monitor fabric activity and switch configuration for all interfaces but does not allow changing configuration or clearing statistics.
All switch interfaces are assigned to both default groups. The admin user has read, use, and configure access to all the interfaces. Any user added to the read-only group has read-only access to all the interfaces.
You can define a new group, in which case a user added to the group will be a restricted user with privileges to view and use only the interfaces assigned to the group.
Using the GUI to Manage User Accounts & Security Groups
To add a user to a group or to add a group, complete the following steps.
Add users from the main Security page or the following page when selecting Security > Users from the DMF GUI main menu.
Figure 2. Security > Users
Select Actions > + Add User in the Users section.
The system displays the Add User dialog.
Figure 3. Add User
Click Submit.
To edit a user, click the Actions > Edit icon for the requisite Username.
To delete a user, select Actions > Delete Selected Users.
To add a Security Group, select Security > Groups in the Groups section.
Figure 4. Security Groups
Select Actions > + Create Group.
The system displays the Create Group menu.
Figure 5. Create Group
Enter a name for the group and all other required group settings, including:
Category Access
Default - Inherit, Does Not Elevate, Read Only, Read Write (drop-down)
AAA - Inherit, Does Not Elevate, Read Only, Read Write (drop-down)
App Log - Inherit, Does Not Elevate, Read Only, Read Write (drop-down)
Sysops - Inherit, Does Not Elevate, Read Only, Read Write (drop-down)
Time - Inherit, Does Not Elevate, Read Only, Read Write (drop-down)
Telemetry - Inherit, Does Not Elevate, Read Only, Read Write (drop-down)
Enter the show group command to display the currently configured groups and users.
To display the group configuration in the current running-config, enter the show running-config group command, as in the following example:
controller-1# show running-config group
! group
group admin
associate user admin
group dmf-qa
associate switch DMF-FILTER-SWITCH-1 interface ethernet1
associate switch DMF-FILTER-SWITCH-1 interface ethernet2
associate switch DMF-FILTER-SWITCH-1 interface ethernet25
associate switch DMF-FILTER-SWITCH-1 interface ethernet26
associate user test1
group read-only
To use the CLI, enter the following commands from config mode.
controller-1(config)# user bob
controller-1(config-local-user)# full-name Robert Smith
controller-1(config-local-user)# password
Password:
Re-enter:
controller-1(config-local-user)# group admin
controller-1(config-group)# associate user bob
controller-1(config-group)# show user
#User nameFull nameGroups
- |--------- |------------- |--------- |
1adminDefault adminadmin
2bobRobert Smith admin
Enter the no user command from config mode to delete a user, as in the following example.
node1(config)# no user bob
controller-1(config)# show user
#User nameFull nameGroups
- |--------- |------------- |------- |
1adminDefault adminadmin
To add a read-only user, run the following commands from config mode.
controller-1(config)# user john
controller-1(config-local-user)# full-name John Smith
controller-1(config-local-user)# password
Password:
Re-enter:
controller-1(config-local-user)# group read-only
controller-1(config-group)# associate user john
controller-1(config-group)# show user
#User nameFull nameGroups
- |--------- |------------- |--------- |
1adminDefault adminadmin
2bobRobert Smith admin
3john John Smith read-only
Changing User Passwords
To change the password for the admin user or other user accounts, select the Actions > Edit icon for the requisite Username from the Security > Users page.
Figure 6. Security Users
Note: Entering a new password will overwrite the existing one. Leaving it blank retains the original password.
Figure 7. Change Password
Type the new password, confirm it, and click Submit.
Password Reset
Resetting the Password for the Recovery User
To reset the password for the recovery user, please follow one of the following procedures. Perform these steps on each cluster Controller as resetting the password of the recovery user on one Controller won't change it for the recovery user on the other Controller.
Using the Controller’s bash:
Go to the Controller bash by executing debug bash.
Execute sudo passwd
recovery.
admin@controller-1:~$ sudo passwd recovery
New password:
Retype new password:
passwd: password updated successfully
admin@controller-1:~$
From the recovery login account login:
For this to work, the current password for the recovery user is required.
recovery@controller-1:~$ passwd recovery
Changing password for recovery.
Current password:
New password:
Retype new password:
passwd: password updated successfully
recovery@controller-1:~$
Using the API /api/v1/rpc/controller/os/action/system-user/reset-password:
The API call below will reset the recovery user's password to AdminAdmin. The example below uses curl initiated from a Linux host, but any RESTful API client can call the API.
Resetting Password for admin and Other Local Users
To reset the password for admin and other local users, log in to the Controller using recovery user credentials. Use the floodlight-reset-password command to reset the user’s password.
The example below example resets the admin user’s password.
recovery@controller-1:~$ floodlight-reset-password --user admin
Enter new admin password:
Re-enter new admin password:
Password updated for user admin
recovery@controller-1:~$
The example below resets the password for the guest account, a read-only group user.
recovery@controller-1:~$ floodlight-reset-password --user guest
Enter new guest password:
Re-enter new guest password:
Password updated for user guest
recovery@controller-1:~$
Authentication with a User Token and REST API
Use the access-token command to create a long-lived token for authentication with external scripting, such as RESTful API. The token can be deleted (repudiated) at any time. DMF preserves a hashed version of the token in the running-config.
The following example shows an access token with the key sam assigned to the user fred.
controller-1(config)# user fred
controller-1(config-user)# access-token sam
access-token : Y9yVwLawjJ03lSthnBKVh3XeplaJ6sSE
The system replies with the session cookie for use in the REST API.
To display the hashed version of the session cookie, enter the show this command or show running-config fred.
controller-1(config-user)# show this
! user
user fred
access-token sam 459d1d5e0bc42c5bbbee091a984e2807592d11d8a5d4cbac110d32da3f0e436c 2018-01-
08T15:44:02.353Z
hashed-password
method=PBKDF2WithHmacSHA512,salt=N9E9jaPIFTc_ZO9Oq0gd4A,rounds=25000,ph=true,
7eCf1PGAYqUw53vJ2bsGpSVEU5-D6ix5sHufFUi3gFr3AHB4Jqj2eLNZzoo66y_qIRFOOOL8nc5oG5i1gJ1FxA
controller-1(config-user)#
The session cookie does not appear in the running-config; the hashed session cookie instead appears. Because the cookie is in the running-config, it persists over upgrade. To remove the token, enter the no access-token command, as in the following example:
controller-1(config-user)# no access-token sam
Configuring AAA to Manage DMF Controller Access
Authentication, Authorization, and Accounting (AAA) is a general standard for controlling and auditing access to network resources. Various implementations are possible using technologies such as LDAP, RADIUS, Kerberos, TACACS+, or a local database. The current version of DANZ Monitoring Fabric (DMF) supports AAA using a local database on the Controller node or with a remote TACACS+ or RADIUS server. All authentication, authorization, and accounting functions are set to local by default.
Authentication and authorization occur only once, and whatever privileges are associated with the first account found are used. Configuring options separately for authentication and authorization is possible, but Arista Networks recommends using the same settings for simplicity.
When one or more remote AAA servers are available, the best practice is to enable remote logging as the primary authorization method. The local database is only used if no remote server responds to the authorization request before the timeout. By default, the timeout is five seconds per server, with a maximum total timeout of 25 seconds for up to five remote servers.
Note: The admin and recovery user accounts are special accounts that cannot be authenticated remotely using RADIUS or TACACS+. These accounts are always authenticated locally to prevent administrative access from being lost in case a remote AAA server is unavailable.
Note: If the user fails to be authorized by any AAA server, the user account can be authorized as a member of the default group and chosen from the Default Group selection list. The options are admin or read-only when configuring the latter group.
DMF supports AAA services using RADIUS and TACACS+ servers using up to four servers of each type. The default timeout for each AAA server is five seconds, which was previously 30 seconds in earlier releases. The total aggregate timeout for all configured servers is 20 seconds.
By default, the Controller waits five seconds before trying the next server or falling back to the local services provided by the Controller database if that is the configuration. You can change the default timeout, but the maximum aggregated time for all the servers is 25 seconds. Five servers are supported only with a default timeout of five seconds or less.
Using the GUI - Enabling Remote AAA Services
GUI Procedure
To manage the configuration of AAA settings on the DMF Controller, select Maintenance > AAA.
Figure 8. AAA Settings Page
AAA configuration tabs and features include:
Accounting
Authentication
Authorization
RADIUS
TACACS+
Audit Logging
This page supports identifying up to five destination TACACS+ or RADIUS servers, configuring the server secret, and specifying a timeout.
Accounting
To log accounting log messages to a remote server, click Accounting > Edit in the Configuration section.
Figure 9. Accounting Settings
By default, DMF saves accounting logs on the local Controller. If local logging is disabled, these logs will not be available for analysis locally. To send the accounting logs to a remote server, select the Order preference:
Local Only - Default.
Remote Only - Enter a Remote Method, TACACS or RADIUS from the drop-down list.
Local Before Remote - Enter a Remote Method, TACACS or RADIUS from the drop-down list.
Remote before Local - Enter a Remote Method, TACACS, or RADIUS from the drop-down list.
Click Submit.
Tip: Use the Reset button to return to the default setting, if required.
Authentication
To configure the authentication method, click Authentication > Edit in the Configuration section.
Figure 10. Authentication Settings
Select the Authentication Order preference:
Local Only - Default.
Remote Only - Enter a Remote Method, TACACS or RADIUS from the drop-down list.
Local Before Remote - Enter a Remote Method, TACACS or RADIUS from the drop-down list.
Remote before Local - Enter a Remote Method, TACACS or RADIUS from the drop-down list.
Lockout Policy Mode
Configuration for the time-based user lockout policy. In no-lockout mode, the configuration of this container is ignored, and lockout is disabled. In lockout-remote mode, this container's values (or, in their absence, the defaults) dictate the lockout behavior.
Select the desired Lockout Policy Mode:
No Lockout
Lockout Remote
Click Next and set the Password Compliance Requirements from the drop-down list:
None
NIST-800-63B
Custom-Check - Enter the requirements for the detailed password (min/max, etc.).
No-Check
Click Submit.
Authorization
To use a remote server for authorization, click Authorization > Edit in the Configuration section.
Select the Authorization preference:
Figure 11. Authorization Settings
To configure the audit logging method, click Audit Logging > Edit in the Configuration section.
Figure 12. Audit Logging
Set the type of audit logging of request leaf values; if no value is specified, request leaf values will not be audit logged.
Select the Log Request Leaf Values from the drop-down list:
None - Default.
Record All Request Values
Click Submit.
Using the CLI - Enabling Remote AAA Services
CLI Procedure
Use the aaa authorization role default command to assign a default role to the current account. DMF uses the default group if authentication on a remote server is successful but no role is specifically assigned. This command does not apply to local authorization or authentication. If a local user is not associated with a group on the Controller, login is not allowed.
Note: Use the authorization role default admin command carefully because the effect is to provide every user account that authenticates successfully on a remote server with admin-level privileges unless the user account is specifically assigned to a different group.
Time-based User Lockout
Starting in the DMF 8.0 release, DANZ Monitoring Fabric supports time-based user lockout functionality. Users are locked out for ‘t2’ time when attempting with ‘n’ incorrect passwords within ‘t1’ time.
Locked-out users must either be cleared of lockout or wait for the lockout period to expire before attempting another login with a correct password. The feature is disabled by default.
To enable, use the following command:
aaa authentication policy lockout failure <number of failed attempts> window <within t1 time>
duration <lockout for t2 time>
The value range for ‘failure’ is 1 to 255.
The value range for ‘window’ and ‘duration’ is 1 to 4294967295 seconds (2^32-1).
The example below locks out any user out for 15 minutes when attempting three incorrect logins within 3 minutes.
This feature affects only remote logins such as SSH/GUI/REST API using username/password. Console-based login and password-less authentications such as SSH keys, Single Sign-on, and access tokens are unaffected. Locked-out users can still access the Controller via console/password-less authentication.
The feature is node-specific concerning the functionality, i.e., if user1 is locked out of accessing the active Controller in the cluster, they can log in to the standby Controller with the correct password and vice versa. Lockout user information is not persistent across Controller reboot/fail-over.
To view if a user is locked out, admin-group users can run the following command:
show aaa authentication lockout
controller-1# show aaa authentication lockout
User name Host Failed LoginsLockout DateLockout Expiration
--------- |------------ |------------- |------------------------------|------------------------------ |
admin 10.240.88.19312020-09-08 16:07:36.283000 PDT2156-10-15 22:35:51.283000 PDT
To clear the lockout for a user, admin-group users can run the following command:
clear aaa authentication lockout user <username>
To clear all the locked-out users use the following command:
clear aaa authentication lockout
The following example shows how to clear a locked-out admin user:
controller-1# clear aaa authentication lockout user admin
controller-1# show aaa authentication lockout
None.
A Recovery user will also be locked out if attempting with incorrect passwords.
To check if the user is locked out, use the ‘pam_tally2’ tool:
Note: The window parameter does not apply to the Recovery user login as the ‘pam_tally2’ tool does not support it.
Using TACACS+ to Control Access to the DMF Controller
Use remote Authentication, Authorization, and Accounting (AAA) services employing a TACACS+ server to control administrative access to the switch CLI. The following table lists the accepted Attribute-Value (AV) pairs:
Attributes
Values
BSN-User-Role
admin
read-only
bigtap-admin
bigtap-read-only
Note: The remotely authenticated admin and bigtap-admin users and the read-only and bigtap-read-only users have the same privileges. The bigtap-admin and bigtap-read-only values are supported to allow the creation of DMF-specific entries without affecting the admin and read-only TACACS+ server entries.
A remotely authenticated admin user has full administrative privileges. Read-only users on the switch must be remotely authenticated. Read-only access is not configurable for locally authenticated user accounts.
Read-only users can only access login mode, from where they can view most show commands, with some limitations, including the following:
TACACS, SNMP, and user configuration are not visible to the read-only user in the output from the show running-config command.
show snmp, show user, and show support commands are disabled for the read-only user.
Note: Local authentication and authorization take precedence over remote authentication and authorization.
Configure privileges on the remote TACACS+ server using the following attribute-value pairs:
Supported attribute name: BSN-User-Role
Supported attribute values: admin, read-only,
Use the TACACS+ server to maintain administrative access control instead of using the Controller's local database. However, it is best practice to maintain the local database as the secondary authentication and authorization method in case the remote server becomes unavailable.
DANZ Monitoring Fabric requires the following configuration on TACACS+ servers in addition to the configuration required on the Controller.
Authentication Method
Configure the TACACS+ server to accept ASCII authentication packets. Do not use the single connect-only protocol feature.
The DANZ Monitoring Fabric TACACS+ client uses the ASCII authentication method. It does not use PAP.
Device Administration
Configure the TACACS+ server to connect to the device administration login service.
Do not use a network access connection method, such as PPP.
Group Memberships
Create a bigtap-admin group. Make all DANZ Monitoring Fabric users part of this group.
TACACS+ group membership is specified using the BSN-User-Role AVPair as part of TACACS+ session authorization.
Configure the TACACS+ server for session authorization, not for command authorization.
Note: Specify the BSN-User-Role attribute as Optional in the tac_plus.conf file to use the same user credentials to access ANET and non-ANET devices.
Using the GUI to Add a TACACS+ Server
To identify a TACACS+ server to provide remote AAA services, complete the following steps. Repeat this procedure to identify up to five servers.
Procedure
Note: To set the secret and timeout values for all the TACACS+ servers, click the controls under the Default Settings section. Otherwise, set these values individually for each server.
Figure 13. Maintenance AAA
Click the Add TACACS Server from the Actions menu in the TACACS Servers table.
Figure 14. Create TACACS+ Server Dialog
Note: Do not use the pound character (#) in the TACACS secret, as it will be interpreted as the start of a comment in the PAM config file.
Enter the IP address of the TACACS+ server.
Type the password required to access the server in the Secret field.
Note: Click the lock icon to encrypt the password if plain-text passwords are not used in the AAA environment.
Click Submit.
Using the CLI to Enable Remote Authentication and Authorization on the DMF Fabric Controller
CLI Procedure
Use the following commands to configure remote ‘login’ authentication and authorization. The examples use the SSH default for connection type.
controller-1(config)# tacacs server host 10.2.3.201
controller-1(config)# aaa authentication login default group tacacs+ local
controller-1(config)# aaa authorization exec default group tacacs+ local
As a result, all users in the bigtap-admin group on TACACS+ server 10.2.3.201 have full access to the DANZ Monitoring Fabric Controller.
Using the CLI to Add a TACACS+ Server
To view the current TACACS+ configuration, enter the show
running-config tacacs command.
Complete the following steps to configure the DMF Controller with TACACS+ to control administrative access to the switch.
Identify the IP address of the TACACS+ server and any key required for access using the tacacs server command with the following syntax:
Enable up to four AAA servers by repeating this command for each server. For example, using a plaintext key, the following command enables TACACS+ with the server running at 10.2.3.4.
controller-1(config)# tacacs server 10.2.3.4 key 0 secret
If the key is omitted, an empty key is used.
Note: Do not use the pound character (#) in the TACACS+ secret, as it will be interpreted as the start of a comment in the PAM config file.
Encrypt each TACACS+ server connection using a pre-shared key. To specify a key for a specific host, use one of the following:
controller-1(config)# tacacs server host <ip-address> key <plaintextkey>
controller-1(config)# tacacs server host <ip-address> key 0 <plaintextkey>
controller-1(config)# tacacs server host <ip-address> key 7 <plaintextkey>
Replace plaintextkey with a password up to 63 characters in length. This key can be specified either globally, or for each individual host. The first two forms accept a plaintext (literal) key, and the last form accepts a pseudo-encrypted key, such as that displayed with show
running-config.
The following is an example using the key 7 option followed by the encrypted string:
controller-1(config)# tacacs server 10.2.3.4 key 7 0832494d1b1c11
To configure a global key, use the following command:
controller-1(config)# tacacs server key 0 secret
The global key value is used if no key is specified for a given host. If no key is specified globally and no key is specified for a given host, then an empty key is assumed.
Note: Be careful configuring TACACS+ to avoid disabling access to the DANZ Monitoring Fabric (DMF) Controller.
Configuring device-specific TACACS+ server parameters overriding that of the global TACACS+ servers is possible. This applies to switches, service nodes, and recorder nodes. The configuration must be made from the config-device submode. An example to configure a switch-specific TACACS+ server is described below:
Similarly, TACACS+ server keys and timeout values can also be overridden:
controller-1(config)# switch DMF-DELIVERY-SWITCH-1
controller-1(config-switch)# tacacs override-global
controller-1(config-switch)# tacacs server timeout 8
controller-1(config-switch)# tacacs server key 0 qwerty
To move back to using the globally defined TACACS+ servers, run the no tacacs override-global command at the config-switch submode.
To view the TACACS+ configuration on a specific switch, use the show effective-config switch switch-name tacacs command:
controller-1(config-switch)# show effective-config switch DMF-DELIVERY-SWITCH-1 tacacs
! switch
switch DMF-DELIVERY-SWITCH-1
tacacs server host 1.1.1.1 key 7 020700560208
tacacs server timeout 8
The TACACS+ key value displays as a type7 secret instead of plaintext.
Setting up a Tac_Plus Server
After installing the tac_plus server, complete the following steps to set up authentication and authorization for the DMF Controller with the TACACS server:
Configure users and groups.
In the /etc/tacacs/tac_plus.conf file, specify the user credentials and group association.
# user details
user = user1 {
member = anet-vsa-admin
login = des a9qtD2JXeK0Sk
}
Configure the groups to use one of the AV pairs supported by the DMF Controller (for example, BSN-User-Role="admin" for admin users).
# group details
# ANET admin group
group = anet-vsa-admin {
service = exec {
BSN-User-Role="admin"
}
}
# ANET read-only group
group = anet-vsa-read-only {
service = exec {
BSN-User-Role="read-only"
}
}
Note: Different TACACS servers need different ways to define the attributes. The following is an example of configuring an Aruba Clearpass server.
Configure the TACACS+ server and AAA on the DMF Controller.
tacacs server host <IP address> key server’s secret>
aaa authentication login default group tacacs+ local
aaa authorization exec default group tacacs+ local
aaa accounting exec default start-stop locals group tacacs+
This configuration allows authentication and authorization to connect to the TACACS+ server to verify user credentials and privileges. Checking the user account locally only occurs when the remote server is unreachable. In this example, accounting is set to store audit logs locally and send them to the remote server.
Refer to your AAA server documentation for further details or instructions for setting up other servers.
Using the Same Credentials for DMF and Other Devices
To use the same user credentials to access DMF and a non-DMF device, the BSN-User-Role attribute must be specified as Optional in the tac_plus.conf file, as shown in the following example.
group = group-admin {
default service = permit
service = exec {
optional BSN-User-Role = "admin"
}
}
RBAC-Based Configuration for Non-Default Group User
To create an RBAC configuration for a user in a non-default group, complete the following steps:
Create a group AD1.
group AD1
Do not associate any local users.
Use the same group name on the TACACS+ server and associate a user to this group.
Note: The attribute should be BSN-User-Role, and the value should be the group-name.
The following is an example from the open TACACS+ server configuration.
group = AD1 {
service = exec {
BSN-User-Role="AD1"
}
}
After you create the group, associate a user to the group.
user = user3 {
member = AD1
login = cleartext user3
}
Using RADIUS for Managing Access to the DMF Controller
By default, the Authentication and Authorization functions are set to local while the Accounting function is disabled. The only supported privilege levels are as follows:
admin: Administrator access, including all CLI modes and debug options.
read-only: Login access, including most show commands.
Note: RADIUS does not separate authentication and authorization, so be careful when authorizing a user account using a remote RADIUS server to use the correct password configured for the user on the remote server.
The admin group provides full access to all network resources, while the read-only group provides read-only access to all network resources.
Note: The admin and recovery user accounts cannot be authenticated remotely using RADIUS. These accounts are always authenticated locally to prevent administrative access from being lost in case a remote AAA server is unavailable.
DANZ Monitoring Fabric also supports remote AAA server (RADIUS) communication. The following summarizes the options available for each function:
Accounting: local, local and remote, or remote.
Authentication: local, local then remote, remote then local, or remote.
Authorization: local, local then remote, remote then local, or remote.
Note: Fallback to local authentication occurs only when the remote server is unavailable, not when authentication fails.
Configure privileges on the remote RADIUS server using the attribute-value pairs shown in the following table:
Supported attribute names
Supported attribute values
BSN-User-Role
admin
bigtap-admin read-only
bigtap-read-only
The BSN-AV-Pair attribute sends CLI command activity accounting to the RADIUS server.
Using the GUI to Add a RADIUS Server
To identify a RADIUS server to provide remote AAA services, complete the following steps. Repeat this procedure to identify up to five servers.
GUI Procedure
Select Maintenance > AAA > RADIUS from the GUI.
Figure 15. Maintenance AAA: Radius Section
Click the Add RADIUS Server from the Actions menu in the RADIUS Servers table.
Figure 16. Create RADIUS Server Dialog
Enter the IP address of the RADIUS server.
Type the password required to access the server in the Secret field.
Note: Click the lock icon to encrypt the password if plain-text passwords are not used in your AAA environment.
Click Submit.
Using the CLI to Add a RADIUS Server
Use the following command to identify the remote RADIUS server:
For example, the following command identifies the RADIUS server at the IP address 192.168.17.101.
controller-1(config)# radius server host 192.168.17.101 key admin
You can enter this command up to five times to identify multiple RADIUS servers. The Controller tries to connect to each server in the order they are configured.
Setting up a FreeRADIUS Server
After installing the FreeRADIUS server, complete the following steps to set up authentication and authorization for the DMF Controller with the RADIUS server.
Procedure
Create the BSN dictionary and add it to the list of used dictionaries.
Note: Make sure that the BSN-User-Role attribute is in the first position in the Radius attribute dictionary, followed by BSN-AVPair.
Include the arista dictionary in the radius dictionary file: /usr/share/freeradius/dictionary
$INCLUDE dictionary.arista
Configure a sample user with admin and read-only privileges.
The following is an example that defines and configures a user, opens the user file /etc/freeradius/users, and inserts the following entries:
Note: This example shows how the VSA is associated with the user and its privileges. In an actual deployment, a database and encrypted password are necessary.
Configure the RADIUS server and AAA on the DMF Controller.
radius server host <IP address> key server’s secret>
aaa authentication login default group radius local
aaa authorization exec default group radius local
aaa accounting exec default start-stop group radius local
This configuration allows authentication and authorization to connect to the RADIUS server to verify user credentials and privileges. AAA fallback to local occurs only when the remote server is unreachable. In this example, accounting is set to store audit logs locally and send them to the remote server.
Add the DMF Controller subnet to the allowed subnets (clients.conf) on the RADIUS server.
This entry is required if access to the RADIUS server is limited to allowed clients/subnets. The following is an example of the clients.conf file:
Restart the FreeRADIUS service on the server to enable the configuration.
The following is an example accounting record sent from the Controller to the RADIUS server after adding the BSN-AVPair attribute to the /usr/share/freeradius/dictionary.arista file.
Use the custom admin roles feature, which allows the definition of a group that grants the privilege to read or write like a built-in admin group, but only for specific use cases/categories. Configure the required privileges for each predefined set of categories.
This configuration allows a user to define a more flexible group role, such as a group that will enable its users to read and write most of the general configuration like admin, but not the AAA-related parts of the configuration.
This approach helps to create user profiles on DANZ Monitoring Fabric with limited admin access restricted only for the required features.
Categories
Users can be created and associated to groups with the following categories:
category:AAA
Configuration and state related to AAA, including but not limited to local user management, group management AAA group creation and user association.
Controller-1> enable
Controller-1# configure
Controller-1(config)# group aaa-mgmt-group
Controller-1(config-group)# permission category:AAA privilege read-write
Controller-1(config)# user aaa-user
Controller-1(config-user)# password <user_configured_password>
Controller-1(config-user)# group aaa-mgmt-group
Controller-1(config-group)# associate user aaa-user
AAA group user sample configuration
AAA group user sample configuration
Controller-1(config)# aaa authentication login default local
Controller-1(config)# tacacs server host <REMOTE_TACACS_SERVER_IP>
Controller-1(config)# show group aaa-mgmt-group configured-permission
# Permission Privilege
1 category:AAA read-write
Controller-1(config)# show group aaa-mgmt-group effective-permission
#Effective-permission Inferred privilege
- |-------------------------- |------------------------- |
1category:DEFAULT inferred-does-not-elevate
2category:DEFAULT/SENSITIVE inferred-does-not-elevate
3category:AAA read-write
4category:AAA/SENSITIVE does-not-elevate
5category:APPLOGinferred-does-not-elevate
6category:SYSOPSinferred-does-not-elevate
7category:SYSOPS/SENSITIVEinferred-does-not-elevate
8category:TIMEinferred-does-not-elevate
9category:TIME/SENSITIVEinferred-does-not-elevate
When AAA group user tries to configure sensitive information
category:AAA/SENSITIVE:
Sensitive data included in configuration and state related to AAA, including but not limited to local user management, group management.
AAA/SENSITIVE group creation and user association
Controller-1> enable
Controller-1# configure
Controller-1(config)#
Controller-1(config)# group aaa-sen-mgmt-group
Controller-1(config-group)# permission category:AAA/SENSITIVE privilege read-write
Controller-1(config)# user aaa-sen-user
Controller-1(config-user)# password <user_configured_password>
Controller-1(config-user)# group aaa-mgmt-group
Controller-1(config-group)# associate user aaa-sen-user
Controller-1(config)# show group aaa-sen-mgmt-group configured-permission
# Permission Privilege
1 category:AAA/SENSITIVE read-write
Controller-1(config)# show group aaa-sen-mgmt-group effective-permission
# Effective-permission Inferred privilege
1 category:DEFAULT inferred-does-not-elevate
2 category:DEFAULT/SENSITIVE inferred-does-not-elevate
3 category:AAA read-write
4 category:AAA/SENSITIVE inferred-read-write
5 category:APPLOG inferred-does-not-elevate
6 category:SYSOPS inferred-does-not-elevate
7 category:SYSOPS/SENSITIVE inferred-does-not-elevate
8 category:TIME inferred-does-not-elevate
9 category:TIME/SENSITIVE inferred-does-not-elevate
Controller-1(config)#
AAA group user sample configuration
Controller-1(config)# tacacs server host 10.240.176.159 key 12323243
Controller-1(config)#
category:TIME:
Configuration and state related to system time and NTP. TIME group creation and user association
Controller-1> enable
Controller-1# configure
Controller-1(config)#
Controller-1(config)# group time-mgmt-group
Controller-1(config-group)# permission category:TIME privilege read-write
Controller-1(config)# user time-user
Controller-1(config-user)# password <user_configured_password>
Controller-1(config-user)# group time-mgmt-group
Controller-1(config-group)# associate user time-user
Navigate to the following page to create the custom admin groups from GUI.
Login to DMF.
Figure 17. Group Creation
Proceed to Security > Groups and add the required settings.
Figure 18. Custom Group
Permissions & Privileges
Privileges are the permissions set of each category.
Read Only: Grants the user privileges to read only the configurations for the enabled category.
Read-Write: Grants the user read and write privileges for configurations for the enabled category.
Does Not Elevate: The user can neither read nor write configurations for that category. Any category, such as AAA, TIME, etc., configured with Does Not Elevate will not inherit from the parent category.
Inherit: Categories will inherit their permission from their parent category. The DEFAULT category is considered the root of all categories.
For example, if TIME/SENSITIVE is set to inherit and TIME has read-only, then users associated with this group will inherit from the parent category, and it will effectively have the privilege of read-only for TIME/SENSITIVE.
Similarly, if AAA is set to inherit and DEFAULT has read-only, then users associated with this group can read all configurations related to AAA.
Category Identification
DMF helps identify which feature comes under what category, achieved using the following command.
Syntax: :: category <feature_name>
For example, the AAA feature.
Controller-1# configure
Controller-1(config)#
Controller-1(config)# category aaa authentication login default local
syntax of variant:
aaa authentication login default {local | group {tacacs+ | radius} | local group {tacacs+ | radius}
| group {tacacs+ | radius} local}
no aaa authentication login default [local | group {tacacs+ | radius} | local group {tacacs+ |
radius} | group {tacacs+ | radius} local]
Configure authentication parameters
AAA "aaa authentication login default local"
Although the category can be configured, allow-all read access is enabled for the command
Exact matching commands: 1, similar commands (same prefix): 1
In the above output, the AAA "aaa authentication login default local" line signifies that the above feature comes under the AAA category.
Category: AAA is set as read-write privilege as per the user configuration Category: AAA/SENSITIVE is set as read-write since it inherits from the immediate parent, i.e., Category: AAA which is “read-write”.
Rest of the categories are set as does-not-elevate since they inherit from Category: DEFAULT
Example to configure a user part of all categories except AAA:
Category: DEFAULT is set as “read-write” privilege as per the user configuration. Category: AAA is set as “does-not-elevate” privilege as per the user configuration. Category: AAA/SENSITIVE is set as “does-not-elevate” since it inherits from the immediate parent, i.e., Category: AAA which is “does-not-elevate”. The rest of the categories are set as “read-write” since they inherit from Category: DEFAULT.
Remote Users
With Custom Admin, associate users with the groups with required privileges using remote TACACS+ and RADIUS servers. To do that, follow these steps:
Create a user bns-user1 and a desired password.
Create a group “BSN-User-Role with the required categories and privileges and associate the above user to the group.
Configure the username and password, and the group created steps 1 and 2 on the TACACS/RADIUS servers.
Login to the DANZ Monitoring Fabric (DMF) Controller using the above credential.
Commonly used profiles
User Administration
Users belonging to this group can only manage users and groups.
Controller-1> enable
Controller-1# configure terminal
Controller-1(config)#
Controller-1(config)# group user-administration
Controller-1(config-group)# permission category:AAA privilege read-write
Controller-1(config-group)# permission category:DEFAULT: does-not-elevate
Controller-1(config)# user time-user
Controller-1(config-user)# password <user_configured_password>
Controller-1(config-user)# group time-mgmt-group
Controller-1(config-group)# associate user time-user
This chapter describes how to manage SNMP services on a DANZ Monitoring Fabric (DMF) Controller.
SNMP Overview
SNMP provides a method for communication between an NMS or other client and agents (servers) on network devices, which send reports, called traps, regarding their operation and configuration. An SNMP agent manages and organizes the information as a collection of objects called MIBs.
In SNMPv3, an engineID identifies the agent (SNMP server), which helps prevent unauthorized SNMPv3 messages, such as traps, from being accepted or intercepted by unauthorized receivers. The engineID of the SNMP agent is required when configuring an SNMPv3 trap receiver to receive messages from an agent, including a DMF Controller or fabric switch.
In DMF, the engineID is auto-generated for the Controller and fabric switches. The engineID of the DMF Controller is configured for the local node. This configuration must be entered separately on the active and standby Controllers. The acceptable practice recommends configuring a different engineID for each Controller.
Using the DMF GUI to Configure SNMP
Complete the following steps to manage or view the DANZ Monitoring fabric (DMF) Controller SNMP configuration. SNMP configuration tabs and features include:
To enable access to SNMP for the Controller, click the link and enter the required fields, ID, Source, and Action (permit), in the Edit Access Control section.
Figure 2. Edit Access Control
Click Submit to continue.
Figure 3. SNMP Enabled
Under Local Configuration, click Edit and enter an Engine ID value, as required.
Figure 4. Edit SNMP Local Configuration
Click Submit to continue.
Tip: Use the Reset button to clear the Engine ID value, if required.
Figure 5. Local Configuration
To enable SNMP traps, select Global Configuration.
Figure 6. Global Configuration
Click Edit and enter the Contact and Location details. Enable Trap Enabled by moving the selector switch to the right.
Figure 7. Edit SNMP Global Configuration
Click Next to continue.
Enter the Trap Host details for the Server and UDP Port (162 by default) using the Provision control (+) button.
Figure 8. SNMP Trap Host
Click Submit. The dashboard displays the information and confirms Trap Enabled.
Figure 9. Global Configuration Trap Enabled
To create a new Community, select the Actions button under Communities and click + Add Community.
Figure 10. Add Community
Select the Permission type (read-only) from the drop-down and enter the Secret.
Figure 11. Add Community Details
Click Submit—the dashboard updates with the Community details.
Figure 12. Communities
To create an SNMPv3 user, select the Actions button under Users and click + Add User.
Figure 13. Add Users
Enter the required information, such as the Name of the user, the Authentication Passphrase for the user, and the Privacy Passphrase. Use the Privacy Protocol drop-down to select Advanced Encryption Standard (AES) or Data Encryption Standard (DES) encryption to encrypt the SNMP messages between the SNMP agent and the manager.
Figure 14. User Details and Encryption
Click Submit to continue—the dashboard updates with the User details.
Figure 15. Users
Configuring SNMP Traps
Complete the following steps to configure the SNMP traps sent to the trap host. SNMP Traps configurations include:
Click Submit. The dashboard displays the Disk Percent value.
Figure 18. Disk Percent
Select Switch Traps on the SNMP landing page.
Figure 19. Switch Traps
Click Edit and enter the Events values:
PSU Status Change in seconds(s).
Fan Status Changein seconds(s).
Link Status Change in seconds(s).
Enable Authentication Failure by moving the selector switch to the right.
Figure 20. Edit SNMP Switch Traps - Events
Click Next to continue.
Enter the Thresholds values:
1-Minute CPU Load Threshold in percentage.
5-Minute CPU Load Threshold in percentage.
15-Minute CPU Load Threshold in percentage.
Percent Idle in percentage.
Percent Utilization in percentage.
Memory Free in bytes.
Full-Match Flow Table in percentage.
Figure 21. Edit SNMP Switch Traps - Thresholds
Click Next to continue.
Enter the Thermal values:
Min in degrees Celsius.
Max in degrees Celsius.
Interval in seconds(s). It must be equal to or greater than 10.
Status from the drop-down list (None, All, Failed, Good, Missing).
Figure 22. Switch Traps - Thermal
Click Submit. The dashboard displays the Switch Traps values.
Figure 23. Switch Traps Values
Configuring the System Name
Complete the following steps to configure the SNMP system name in the local configuration to a desired value such as a fully qualified domain name (FQDN).
Select Local Configuration on the SNMP landing page.
Figure 24. Local Configuration
Click Edit and enter the chosen System Name string.
Click Submit. The dashboard displays the new System Name string.
Using the CLI to Configure SNMP
This section describes using the CLI to configure and manage SNMP settings for the DMF Controller cluster.
Note: To configure a separate SNMP server for switches or Service Nodes, configure an access list to permit access from required clients.
Configuring SNMP Access to the Controller
By default, SNMP access to the Controller is disabled. The default SNMP access list is empty, meaning access is not permitted unless specifically enabled.
The following commands enable access to the Controller by remote SNMP clients on the specified subnetwork:
Replace ipaddress with the IP address of the host. Replace udp-port with the port number used by the SNMP traps. For example, the following command identifies a management system at 192.168.17.150 using UDP port 162.
UDP port 162 is the default for SNMP trap messages; UPD port 161 is the default port for general SNMP messages.
The following are the SNMP traps generated by the Controller running on a VM or the hardware appliance:
Name OID Trap generation
--------------------------------------------------------------------------
cpuload .1.3.6.2.4.1.2021.10.1.5.1 when load (average over 1 minute) > %90
memtotalfree .1.3.6.2.4.1.2021.4.11.0 when freemen (of entire Linux OS) < 50K
The following are the SNMP traps generated only by the hardware appliance:
cputemp .1.3.6.2.2.1.99.1.1.1.4.1001 when CPU core temp > vendor
specified threshold value
ambienttemp .1.3.6.2.2.1.99.1.1.1.4.2001 when chassis inlet temp >
vendor specified threshold value
powersupply .1.3.6.2.2.1.99.1.1.1.4.3001 when power consumption >
vendor specified threshold value
fan**speed .1.3.6.2.2.1.99.1.1.1.4.40** when fan speed < vendor
specified threshold
Configuring disk-percent trap will monitor the root partition and the /var/log partition. To configure the trap:
controller-1(config)# snmp-server trap
disk-percent set logging partition space use percentage at which to send trap
<disk-percent> Percent disk utilization (1..100)
controller-1(config)# snmp-server trap disk-percent 75
The following is the entry created in the /etc/snmp/snmpd.conf file when you configure the trap on the DMF controller:
To set the SNMP community string, which is a password used by a management application for accessing SNMP information, enter the snmp-server community command from config mode, as in the following example:
Note: Even though the CLI has options for ro or read-only and rw or read-write types of community strings, DANZ Monitoring Fabric supports only the ro option.
controller-1(config)# snmp-server community ro <string>
This command sets the community string for read-only access to the SNMP trap server.
Note: To push the SNMP trap host configuration to the monitoring switches, configure the community string to access the MIBs on the controller and switches. The SNMP trap server uses the same community string to receive and process the traps.
To set the SNMP location, enter the snmp-server location command from config mode, as in the following example:
To view the current SNMP configuration, enter the show running-config snmp command.
Note: The community string appears as a Type 7 encoded value in the running-config.
To monitor the Controller’s /var/log and root partitions, configure the following trap:
disk-percent percent: Replace percent with the percentage that triggers a trap when exceeded.
Note: Configuring the disk-percent trap on the Analytics Node will monitor the /var/lib/analytics/data folder, the /var/log folder, and the root partition.
To set the SNMP system name string to a desired value such as a fully qualified domain name (FQDN), the Controller configuration must be updated in the local node mode and hence must be done individually for each Controller node in the cluster.
Note: The same configuration applies also to Service Nodes, Recorder Nodes, and Analytics Nodes.
You can enter a chosen string as shown in the example below:
controller-2# conf
controller-2(config)# local node
controller-2(config-local)# snmp-server
engine-id Value for the SNMP engine ID, a text string up to 27 characters long
system-name SNMP system name to expose (sysName)
DMF-MACSEC-2(config-local)# snmp-server system-name
<System-name><String>
controller-2(config-local)# snmp-server system-name DMF-C2.aristanetworks.com
controller-2(config-local)# end
controller-2# show run local
! local
local node
hostname controller-2
snmp-server system-name DMF-C2.aristanetworks.com
interface management
!
ipv4
ip 10.240.189.233/27 gateway 10.240.189.225
method manual
dns search qa.bsn.sjc.aristanetworks.com
dns server 10.240.48.6
!
ipv6
method manual
Without the above configuration, an snmpwalk command would return the hostname in the sysName parameter, as shown in the following:
controller-2# conf
controller-2(config)# local node
controller-2(config-local)# no snmp-server system-name DMF-C2.aristanetworks.com
controller-2(config-local)# end
controller-2#
Use the following keywords with the snmp-server switch trap command as required.
auth-fail: Sends a trap when an authentication attempt fails.
cpu-loadcpu-load: Replace cpu-load with the threshold for CPU utilization.
fan-status: Sends a trap when the fan status changes. Set the interval for monitoring between 10 and 100,000 seconds.
fm-flow-table-utilutil: Replace util with the percentage that triggers a trap when exceeded.
link-status: Sends a trap when the status of a link changes. Set the interval for monitoring between 1 and 100,000 seconds.
mem-freemem-free: Replace mem-free with the threshold (in bytes) for memory utilization.
percent-idlepercent: Replace percent with the percentage of CPU idle utilization that triggers a trap when exceeded.
percent-utilizationpercent: Replace percent with the with the percentage of CPU utilization that triggers a trap when exceeded.
psu-status: Generate a trap when PSU status changes. Set the interval for monitoring between 10 and 100,000 seconds.
thermal: Sends a trap when the thermal sensor status changes as specified using the following options.
all: Includes failed, good, and missing.
failed: Sends a trap when the thermal sensor fails.
good: Sends a trap when the thermal environment is normal.
missing: Sends a trip when the thermal sensor is not present.
interval: Sends the trip after the expiry of the specified interval. The range is 10 to 100,000 seconds.
[ min-temp | max-temp ]: A trap is generated when the temperature in degrees Celsius is less than min-temp or greater than max-temp.
Note: It is highly recommended to use percent-idle or percent-utilization instead of cpu-load trap.
SNMP Traps for DMF Service Node Appliance
The following are the SNMP traps supported by the DANZ Monitoring Fabric (DMF) Service Node appliance.
PSU failed/recovered
Fan failed/recovered
Temp exceeded some threshold or came back to normal
Interfaces up/down
SN inaccessible by the Controller
SN NetFlow GW is inaccessible
Percent (%) packet drop exceeded some threshold
Managing the SNMPv3 Engine ID for Trap Receivers
SNMPv3 adds authentication and encryption to the features provided by earlier versions of SNMP (v1 and v2). DANZ Monitoring Fabric (DMF) supports the SNMPv3 user-based security model (USM) for message security through authentication and encryption.
In SNMPv3, an engineID identifies the agent (SNMP server), which helps prevent unauthorized SNMPv3 messages, such as traps, from being accepted or intercepted by unauthorized receivers. The engineID of the SNMP agent is required when configuring an SNMPv3 trap receiver to receive messages from an agent, including a DMF Controller or fabric switch.
In DMF, the engineID is auto-generated for the fabric switches. To view the engineID for a specific fabric switch, enter the following command:
controller-1> show switch <switch-name> running-config
For the DMF Controller, specify an engine-ID keyword that is used to generate the Controller engine-ID. The engine-ID keyword is a text string, up to 27 characters. To configure the engine-id, use the snmp-server engine-id string command from the config-local-node submode, as in the following example:
controller-1(config)# local node
controller-1(config-local)# snmp-server engine-id controller-1_EngineID
The engineID of the DMF Controller is configured for the local node. This configuration must be entered separately on the active and standby Controllers. The acceptable practice recommends configuring a different engineID for each Controller.
Note: The engine-id configuration is not included when applying a saved running-config to the Controller. The engine-id configuration must be reapplied using snmp-server
engine-id command.
The snmp-server engine-id command sets the engine-ID for the Controller using the following format:
0x80001f8804 + <hex string>
where hex string is the ASCII hex version of the user-supplied string, which can be found using a tool like xxd:
$ echo "abcdef--g" | xxd -ps
6162636465662d2d670a
This command lets you calculate the engine ID, as in the following example.
Use the snmp-server user command in config mode to create a user account for SNMP v3 access. When running an snmpwalk (snmpget, snmpgetnext,
snmpbulkget) from a shell, passphrases should be enclosed in single quotes. Entering the passphrase with double quotes (” “), may result in an error. This command has the following syntax:
auth | auth 0 | auth 7: Use a plaintext passphrase or a type 7 encoded passphrase.
cleartext-passphrase: A cleartext passphrase from 8 to 64 alphanumeric characters including dash (“-” and space). A dash or whitespace is not allowed at the beginning or end of the passphrase. Other special characters are not allowed.
private-passphrase: A type 0 encoded passphrase from 8 to 64 alphanumeric characters including dash (“-”) and space. A dash or whitespace is not allowed at the beginning or end of the passphrase. Other special characters are not allowed.
type-7-passphrase: A type 7 encoded passphrase from 8 to 128 alphanumeric characters including dash (“-”) and space. The maximum text string length that can be used with a Type 7 encoder, which can be found online, is 64. A dash or whitespace is not allowed at the beginning or end of the passphrase. Other special characters are not allowed.
priv {aes | des}: Optional keyword to perform Advanced Encryption Standard (AES) or Data Encryption Standard (DES) encryption of the following passphrase, which is used as an encryption key to encrypt the SNMP messages between the SNMP agent and the manager.
userusername: Up to 32 alphanumeric characters including dash (“-“) and underscore (“_”) Spaces are not permitted. After you configure the username with a plaintext passphrase, the output from the show snmp-server command displays the passphrases in Type7 encoded strings. The Controller's configuration gets pushed through zero touch networking (ZTN) to the connected fabric switches.
Note: DANZ Monitoring Fabric (DMF) only supports the ro or read-only type of community string option.
SNMPv3 Command Examples
Example 1. The snmp_1 user is configured for authentication (authNoPriv) with the plaintext password authauth1.
controller-1(config)# snmp-server user snmp_1 auth authauth1
Example 2. The snmp-2 user is configured for authentication (authNoPriv) with the plaintext password authauth1.
controller-1(config)# snmp-server user snmp-2 auth 0 authauth2
Example 3. The snmp11 user is configured for authentication and DES encryption (authpriv) with the auth password authauth11 and the encryption key privpriv11.
controller-1(config)# snmp-server user snmp11 auth 0 authauth11 priv des 0 privpriv11
Example 4. The snmp21 user is configured for authentication and AES encryption (authpriv) with the auth password authauth21 and the encryption key privpriv21.
The following are examples of Type7 encoded passphrases:
controller-1(config)# snmp-server user snmp1 auth 7 0207114f03071a35441f
controller-1(config)# snmp-server user snmp20 auth 7 0207114f03071a35441c59 priv des 7 021616521d161d285a1c59
controller-1(config)# snmp-server user snmp30 auth 7 0207114f03071a35441d59 priv aes 7 021616521d161d285a1d59
Configuring SNMP on a Specific Switch
Configuring SNMP for a specific switch does not affect the Controller or other switches. Otherwise, the configuration is similar to configuring SNMP at the Controller level, using the Maintenance > SNMP option.
Note: Before configuring SNMP for a specific switch, enable SNMP access to the Controller.
Using the GUI to Configure SNMP on a Specific Switch
To use the GUI to merge/override the default SNMP configuration with switch-specific SNMP configuration, complete the following steps:
Select Fabric > Switches and click the link for a specific switch.
Figure 25. Fabric Switches
On the Switches page, click the Actions control followed by Configure Switch.
Figure 26. Configure Switch Dialog
This page allows merging and overriding the default configuration pushed from the DANZ Monitoring Fabric (DMF) Controller with switch-specific SNMP configuration.
To merge or override the SNMP configuration, click the SNMP link. Choose from the SNMP Settings drop-down to Merge with Global Config or Override Global Config.
Make any changes required to the specific switch configuration, click Next to customize the SNMP traps, or click Submit.
To merge or override the configuration for SNMP traps, click the SNMP Traps link and choose from the SNMP Switch Trap Settings drop-down to either Merge with Global Config or Override Global Config.
Figure 27. SNMP Traps
Make any changes required to the specific switch configuration and click Submit.
Using the CLI to Configure SNMP on a Specific Switch
Note: Before entering SNMP commands from the config-switch submode, enable SNMP access to the Controller.
When using the config-switch submode for a specific switch, configuration changes, including SNMP, do not affect the Controller or other switches. Otherwise, the configuration is similar to configuring SNMP in config mode at the Controller level.
Entering the snmp-server enable traps command in config mode pushes snmp-server enable configuration to each connected fabric switch. Verify the switch configuration by entering the show effective-config switch switch-name snmp from the CLI, as in the following example.
Like the GUI, use the CLI to merge or override the default SNMP configuration with switch-specific SNMP configuration. To do so, complete the following steps:
Add the SNMP configuration at the Controller. This is the default SNMP configuration pushed to all the switches. The following is an example configuration:
controller-1(config)# show running-config snmp
! snmp-server
snmp-server host 10.1.1.1
snmp-server enable traps
snmp-server community ro 7 02161159070f0c
snmp-server contact Alice
snmp-server location 'San Francisco'
snmp-server user user1 auth 7 0217135e191216344541
Configure switch-specific-parameters at the config-switch submode.
In the config-switch submode, type either merge-global to merge the global config with the switch-specific config or override-global to override the global config with the switch config. When choosing neither, the switch inherits the global config, and any configuration added under the config-switch submode will be redundant.
Check the SNMP configuration running on the switch using the CLI command show effective-config switch switch-name snmp:
controller-1(config-switch)# show effective-config switch switch-btsw-1 snmp
! switch
switch switch-btsw-1
snmp-server host 10.1.1.1
snmp-server host 10.1.1.2
snmp-server enable traps
snmp-server community ro 7 02161159070f0c
snmp-server contact Bob
snmp-server location 'San Jose'
snmp-server user user1 auth 7 0217135e191216344541
snmp-server user user2 auth 7 0207175f0d01072b4742
When using merge-global, the effective configuration on the switch is a merge of the global configuration and the switch-specific configuration.
Note: SNMP community, user, and host are of list-type. In merge-mode these list-type configurations append to potentially existing global config.
Below is an example with override-global:
controller-1(config-switch)# snmp-server override-global
controller-1(config-switch)# show effective-config switch switch-btsw-1 snmp
! switch
switch switch-btsw-1
snmp-server host 10.1.1.2
snmp-server contact Bob
snmp-server location 'San Jose'
snmp-server user user2 auth 7 0207175f0d01072b4742
When using override-global, the effective configuration on the switch is only the switch-specific configuration and completely overrides the default configuration inherited from the Controller.
Configuring SNMP traps using merge and override global commands is similar. See examples below:
To limit SNMP access to clients in specific IP subnetworks, enter the snmp-server community command from the config-switch submode on the DMF Controller. This command has the following syntax:
When using the merge-global and override-global commands at the config-switch submode, the SNMP community for the switch can be changed as shown in the following example:
SNMP configuration on the controller:
controller-1(config)# show running-config snmp
! snmp-server
snmp-server host 10.1.1.1
snmp-server community ro 7 02161159070f0c
snmp-server contact Alice
snmp-server location 'San Francisco'
snmp-server user user1 auth 7 0217135e191216344541
SNMP configuration on the switch:
controller-1(config-switch)# show run switch switch-btsw-1
! switch
switch switch-btsw-1
snmp-server override-global
snmp-server enable traps
snmp-server host 10.1.1.2
snmp-server community ro 7 021616521d071b24
snmp-server contact Bob
snmp-server location 'San Jose'
snmp-server user user2 auth 7 0207175f0d01072b4742
SNMP Clear Trap
SNMP trap messages are sent whenever a threshold is reached, or an HW failure happens, like PSU failure/removal. An SNMP clear trap message is sent whenever a threshold is less than the specified range or the HW failure is fixed, such as when the PSU starts working.
There is no command to enable this feature. This feature is automatically enabled when configuring the SNMP trap on the Controller.
SNMP traps that do not have associated clear traps have other ways of notifying state change. For example, link up and link down traps are sent when the link goes up and down. The /etc/snmp/snmpd.conf file lists all SNMP traps and clear trap settings.
Note: SNMP clear traps will be sent without any prior associated SNMP traps when the system comes up or there is any SNMP configuration change. Ignore these SNMP clear traps.
SNMP clear trap messages are not supported on DMF switches running EOS.
The following are switch traps for which clear traps will be sent:
switch trap cpu-load
switch trap fm-flow-table-util
switch trap mem-free
switch trap percent-idle
switch trap percent-utilization
These are the appliance (Controller, Service Node, Recorder Node, Analytic Node) traps for which clear traps will be sent.
Note: Upgrade the appliance IDRAC Firmware to the recommended version of 5.10.50.00 or later.
disk-percent
memtotalfree
lowmemavailable
cpuload
cputemp
cpu1temp
ambienttemp
exhausttemp
powersupply
fanspeed
The number of fans on an appliance varies. Depending on the number of fans on the appliance, fanspeed clear traps are sent.
Fan speed traps are named fan1Aspeed, fan1Bspeed, etc.
psuCount
fanCount
SNMP Trap Generation for Packet Drops and Link Saturation
Users wishing to be notified about packet drops or high link saturation in the DANZ Monitoring Fabric (DMF) fabric can receive SNMP traps for these events.
Specifically, when trap generation is enabled, the following events will send an SNMP trap to the configured trap collector:
Transmit packet drop counter increase at a switch interface managed by DMF.
Traffic saturation levels above the high water mark of 90% on a link managed by DMF.
A drop in traffic saturation levels below the low water mark of 70% if the link was previously saturated.
Note: This feature is compatible with all hardware platforms with DMF 8.4 version or higher. However, this does not include the DMF Managed Analytics, Recorder Node, and Service Node appliances.
SNMP Trap Generation for Packet Drops and Link Saturation introduces the following OIDs:
OID
Type
Description
.1.3.6.1.4.1.37538.68.77.70
none
Root of the tree for custom DMF extensions.
.1.3.6.1.4.1.37538.68.77.70.1
none
Subtree root for link saturation warnings based on data from the endpoint applications/dmf/info/warnings/link-saturation.
.1.3.6.1.4.1.37538.68.77.70.1.1.0
integer
The number of new link saturation warnings that have been detected since the last polling interval (15 seconds).
.1.3.6.1.4.1.37538.68.77.70.1.2.0
integer
The number of link saturation warnings that have cleared since the last polling interval (15 seconds).
.1.3.6.1.4.1.37538.68.77.70.1.3.0
oidref
The OID to the start of the table containing all known link saturation warnings. Always holds the value:
.1.3.6.1.4.1.37538.68.77.70.1.5
This object is sent as a varbind in the trap newLinkSaturationWarnings to indicate to the trap collector that this OID should be used in a walk for the updated list of warnings.
.1.3.6.1.4.1.37538.68.77.70.1.4.0
oidref
The OID to the start of the table containing all cleared link saturation warnings since the last polling interval. Always holds the value:
.1.3.6.1.4.1.37538.68.77.70.1.6
This object is sent as a varbind in the trap linkSaturationWarningsCleared to indicate to the trap collector that this OID should be used in a walk for the updated list of cleared warnings.
.1.3.6.1.4.1.37538.68.77.70.1.5
string
Start/root of the table of link saturation warnings. Holds the string: "Table of current warnings"
SNMP Trap Generation for Packet Drops and Link Saturation OIDs (cont'd):
OID
Type
Description
.1.3.6.1.4.1.37538.68.77.70.1.5.1
string
[Table row 1] A string summarizing a link saturation warning as a dot-separated tuple of switch DPID, interface name, and traffic direction (either ‘RX’ or ‘TX’). Example:
"00:00:52:54:00:a2:d8:9d.ethernet2.TX"
Indicates that port ethernet2 on the switch identified by DPID 00:00:52:54:00:a2:d8:9d has experienced transmit link saturation levels above 90%.
.1.3.6.1.4.1.37538.68.77.70.1.6
string
Start/root of the table of cleared link saturation warnings. Holds the string: "Table of cleared warnings"
.1.3.6.1.4.1.37538.68.77.70.1.6.1
string
[Table row 1] A string summarizing a recently-cleared link saturation warning as a dot-separated tuple of switch DPID, interface name, and traffic direction (either ‘RX’ or ‘TX’). Example:
"00:00:52:54:00:a2:d8:9d.ethernet2.TX"
Indicates that the transmit link saturation warning for port ethernet2 on the switch identified by DPID 00:00:52:54:00:a2:d8:9d has cleared (dropped below the low watermark of 70%).
.1.3.6.1.4.1.37538.68.77.70.2
None
Subtree root for packet drop warnings which are issued when the TX drop counter of an interface increases. Based on the data from the endpoint core/switch/dmf-stats/interface
.1.3.6.1.4.1.37538.68.77.70.2.1.0
integer
The number of interfaces that have experienced an increase in the transmit drop counter since the last polling interval (15 seconds).
.1.3.6.1.4.1.37538.68.77.70.2.2.0
oidref
The OID to the start of the table containing all of the interfaces that have seen its transmit drop counter increase since the last polling interval. Always holds the value:
.1.3.6.1.4.1.37538.68.77.70.2.3
This object is sent as a varbind in the trap newPacketDropsWarnings to indicate to the trap collector that this OID should be used in a walk for the updated list of interfaces with increased counters.
.1.3.6.1.4.1.37538.68.77.70.2.3
string
Start/root of the table of interfaces. Holds the string: "Table of interfaces with incremented drop counters"
.1.3.6.1.4.1.37538.68.77.70.2.3.1
string
[Table row 1] A string representing an interface that has experienced a transmit drop counter increase. Example:
“00:00:52:54:00:a2:d8:9d.ethernet2”
represents interface ethernet2 on the switch identified by DPID 00:00:52:54:00:a2:d8:9d.
.1.3.6.1.4.1.37538.68.77.70.2.3.2
counter64
[Table row 2] An unsigned integer holding the current counter value.
This feature introduces the following traps:
newLinkSaturationWarnings
When new link saturation warnings are generated, in addition to its name, the trap contains the following varbinds:
.1.3.6.1.4.1.37538.68.77.70.1.1.0
.1.3.6.1.4.1.37538.68.77.70.1.3.0
linkSaturationWarningsCleared
When one or more existing link saturation warnings are cleared, in addition to its name, the trap contains the following varbinds:
.1.3.6.1.4.1.37538.68.77.70.1.2.0
.1.3.6.1.4.1.37538.68.77.70.1.4.0
newPacketDropsWarnings
When packet drop counters are seen incrementing on one or more managed interfaces, in addition to its name, the trap contains the following varbinds:
.1.3.6.1.4.1.37538.68.77.70.2.1.0
.1.3.6.1.4.1.37538.68.77.70.2.2.0
When traps are enabled, the Controller can be queried for the monitored data to send the traps. The MIB for this feature is rooted at the OID .1.3.6.1.4.1.37538.68.77.70 . For a Controller at 192.0.2.1 with an SNMP community named example, a query using snmpwalk and the corresponding response may resemble the following:
The tables in the response are empty when there are no warnings:
iso.3.6.1.4.1.37538.68.77.70.1.1.0 = INTEGER: 0
iso.3.6.1.4.1.37538.68.77.70.1.2.0 = INTEGER: 0
iso.3.6.1.4.1.37538.68.77.70.1.3.0 = OID: iso.3.6.1.4.1.37538.68.77.70.1.5
iso.3.6.1.4.1.37538.68.77.70.1.4.0 = OID: iso.3.6.1.4.1.37538.68.77.70.1.6
iso.3.6.1.4.1.37538.68.77.70.1.5 = STRING: "Table of current warnings"
iso.3.6.1.4.1.37538.68.77.70.1.6 = STRING: "Table of cleared warnings"
iso.3.6.1.4.1.37538.68.77.70.2.1.0 = INTEGER: 0
iso.3.6.1.4.1.37538.68.77.70.2.2.0 = OID: iso.3.6.1.4.1.37538.68.77.70.2.3
iso.3.6.1.4.1.37538.68.77.70.2.3 = STRING: "Table of interfaces with incremented drop counters"
Using the CLI to Configure the SNMP Traps
This feature is enabled by enabling trap generation, configuring an SNMP community or USM user for sending the traps, and specifying a trap receiver host, as shown below.
(config)# snmp-server enable traps
(config)# snmp-server community ro example
(config)# snmp-server host 192.0.2.10
Traps are disabled when trap generation is disabled using the following command:
(config)# no snmp-server enable traps
CLI Show Commands
SNMP configurations, including traps, communities, or users, appear in the running-config using the show command.
# show running-config snmp
! snmp-server
snmp-server host 192.0.2.10
snmp-server enable traps
snmp-server community ro 7 02031c5a06160324
The configuration entry snmp-server enable traps indicates that trap support, including this feature, is enabled.
Syslog Messages
SDCACHE7001: Error while refreshing cache for Query{<query>, includedStateTypes=[LOCAL_CONFIG, GLOBAL_CONFIG, OPERATIONAL]}
Generated when the statistics collected and monitored for this feature could not be cached. The message is logged along with a stack trace containing further details.
SDCACHE7002: Query for data failed for Query{<query>, includedStateTypes=[LOCAL_CONFIG, GLOBAL_CONFIG, OPERATIONAL]}
Generated when the periodic query to the Controller for collecting the statistics monitored when evaluating if traps should be sent failed. The message is logged along with a stack trace containing further details.
SNMPEXT4001: Could not disable configuration <configuration>
<configuration> is one of:
dmf_trap_monitors
dmf_warning_extensions
Generated when the SNMP configurations associated with a trap, named [configuration], could not be disabled. The message is logged along with a stack trace containing further details.
SNMPEXT4002: Could not enable configuration <configuration>
<configuration> is one of:
dmf_trap_monitors
dmf_warning_extensions
Generated when the SNMP configurations associated with a trap, named [configuration], could not be enabled. The message is logged along with a stack trace containing further details.
Troubleshooting
If the CLI show command indicates:
Traps are enabled.
A community or user is configured, and
A trap host is configured.
The trap host may not be reachable from the Controller, or statistics collection and monitoring for packet drops or link saturation may need to be enabled correctly. Check the logs for any messages described in the previous section using the following command:
> show logging controller | grep ‘SDCACHE\|SNMPEXT’
Contact This email address is being protected from spambots. You need JavaScript enabled to view it. if any of these logs appear and:
Traps are not sent from a system with known link saturation or packet drop warnings, e.g., via the CLI or GUI.
A query, e.g., snmpwalk for OIDs rooted under 1.3.6.1.4.1.37538.68.77.70 does not result in responses or return errors.
Considerations
The user cannot specify the link saturation trap thresholds. Traps are sent when link saturation levels cross the 90% threshold and are cleared once levels drop below 70%.
The tree associated with this feature is only available through polling with snmpwalk when traps are enabled.
The OIDs do not have resolvable names.
Polling of link and interface states happens at a fixed interval; there may be a several-second delay between the occurrence of a trap-triggering event and sending the trap.
DHCPv4-Based First-boot for DMF Controller and Managed Appliances
This chapter outlines a solution for provisioning DANZ Monitoring Fabric (DMF) appliances via PXE and automating the configuration of the first boot parameters using Ansible.
Introduction
Typically, the deployment of DANZ Monitoring Fabric (DMF) on supported hardware appliances involves two steps:
Installing an appropriate image.
Configuration of firstboot parameters such as IP address (DHCP/Static), DNS and NTP server address, admin password(s), cluster information etc.
In this context, Pave refers to the automation of the first-time installation of a DMF hardware appliance. This involves installing a DMF image on a hardware appliance and completing the first-boot configuration. First-boot configuration uses an Ansible playbook as an automation tool. In contrast, Repave refers to automating the re-installation of DMF images on supported DMF appliances. This involves the automated process of re-installing a DMF image on a DMF hardware appliance and completing the first-boot configuration.
Several prerequisites are required. The production/lab environment should have:
A DHCP server that supports the configuration of DHCP options 66 and 67.
A TFTP server that can serve the bootloader and the corresponding configuration.
An NFS server to serve the net-bootable appliance image.
A server with ansible-playbook and Arista-supported playbook modules installed.
Supported DMF hardware appliances with preset boot order settings and a PXE-enabled management port.
Note: In the case of Repave action, this new feature introduces a new command (boot pxe) to change the boot order to PXE boot. On reboot, the appliance will automatically perform image re-installation.
Prepare Services (TFTP/NFS) for PAVE/REPAVE Operation
The following steps must be completed before deploying the DANZ Monitoring Fabric (DMF) ISO image on a TFTP server.
Create a directory by name images on the TFTP server and copy the ISO image to the TFTP server.
SSH to the server with sudo privileges and create a temporary directory using the command below.
$ mktemp -d /tmp/tmp.2syaj0amL7
Mount the ISO image to the directory created above.
$ mount /images/*.iso /tmp/tmp.2syaj0amL7
Copy the following files to the root TFTPboot directory.
Copy kernel configurations that assist the DMF appliance booting with UEFI mode to locate the bootloader and vmlinuz files from the TFTP and NFS server. Arista recommends defining a name for menu entry so that it is easily remembered and prefixed with appliance type for differentiation, i.e., DCA-DM-CDL-dmf-8.4.0.
Note:In the currently supported UEFI boot mode, Arista recommends creating a PXE configuration file for each hardware MAC address, i.e., instead of the default grub.cfg use grub.cfg-01-xx-xx-xx-xx-xx-xx . Specify the MAC address in all lower-case with : (colon) replaced by - (dash).
To perform the action detailed in the note above, obtain the MAC / Hardware address of the management interfaces. Obtaining the MAC address can be done in two ways, as described below:
If this is the first time the appliance is being installed, either:
Use the Integrated Dell Remote Access Controller (iDRAC) web console to determine the MAC address of the management interface.
Figure 1. Using the iDRAC menu to obtain the Management Interface MAC Address
Enter the device BIOS menu during boot-up and determine the MAC address of the management interface.
Figure 2. Using the BIOS menu to obtain the Management Interface MAC Address
When performing a re-installation of DMF appliances, obtain the MAC address of the appliance via the CLI command.
For DMF Controllers with SKU DCA-DM-C450 or DMF Analytics Nodes with SKU DCA-DM-AN450, execute the following commands on the Controller to obtain the relevant MAC address.
For DMF Controllers with SKU DCA-DM-CDL, including all DMF Recorder Nodes, execute the following commands on the device to obtain the relevant MAC address.
If both the management ports of the appliance are connected to the top of the rack management switch, Arista recommends generating a separate PXE configuration file for both management ports.
cat << EOF | sudo tee /var/lib/tftpboot/boot/grub/grub.cfg-01-0a-0b-0c-0d-0e-0f
set default="0"
loadfont unicode
set gfxmode=auto
insmod all_video
insmod gfxterm
serial --unit=0 --speed=115200
serial --unit=1 --speed=115200
terminal_input console
terminal_input --append serial_com0
terminal_input --append serial_com1
terminal_output gfxterm
terminal_output --append serial_com0
terminal_output --append serial_com1
set timeout=5
menuentry "Install <INSERT_USER_FRIENDLY_NAME>" {
linux dmf-controller/vmlinuz boot=casper netboot=nfs nfsroot=<ip-address of PXE server>
:/srv/install/dmf-controller toram noprompt ip=dhcp -- unattended_installation autofirstboot_
via_ssh
initrd dmf-controller/initrd.lz
}
EOF
DHCPv4-Based First-boot
- DHCPv4-based first-boot for DMF Controller and Managed Appliances
There are two main steps involved in accomplishing the automatic installation of an image on a supported DANZ Monitoring Fabric (DMF) hardware appliance and the completion of the first-boot configuration.
Auto-installation of a DMF image uses well-known services like DHCP, TFTP, NFS, and PXE. DMF images are PXE bootable, and using the services above, perform the auto-installation of images on the DMF appliances.
A high-level procedure for auto-installation is described below.
Configure a DHCP server to provide a DHCP IP address.
Enter the next-server IP address (This is the TFTP server IP address specified by Option 66 configuration on the DHCP server).
next-server <TFTP_SERVER_IP>;
class "pxeclient" {
match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
filename "boot/grub/x86_64-efi/core.efi"; # x86 EFI
}
Arista recommends binding a static IP to the hardware MAC address of the DMF appliance. The following is a sample configuration using a Linux-based DHCP service.
On a DMF HW appliance using PXE boot, enable PXE on the management interface NIC card.
Figure 3. Enable interfaces in BIOS to be PXE bootable
For the initial PAVE action, press the F12 key during the initial boot process to manually trigger the PXE boot. This step is typically not required when reinstalling (REPAVE) the DMF appliance images.
Figure 4. Manual PXE Boot by pressing F12
Power cycle the DMF HW appliance.
The DHCP client on the NIC card sends a DHCP discovery request and gets an IP address and next-server (TFTP server) IP address.
The management NIC gets the bootloader via PXE boot using a TFTP server. A bootloader config file with a filename based on the MAC address of the appliance is configured and saved on the DHCP server.
The appliance obtains configuration parameters like the NFS mount, where appliance ISO images are stored.
The appliance boots from the boot loader and obtains the DMF appliance ISO image from the NFS server.
The appliance boots the ISO image and starts the installation process without user input.
On reboot, the appliance again acquires a DHCP-based management IP address, sets up initial default user credentials, and waits for auto first-boot configuration via Ansible.
Note: For repave, the procedure is the same except that the DMF appliance is already running a DMF image and needs to boot from PXE again for re-installation.
Auto Configuration of First-boot
Auto-configuring of first-boot parameters uses Ansible and is accomplished by interacting with auto-firstboot-cloud-plugin, which is available in hardware appliances.
Begin the initial configuration playbook in Ansible. The following is a sample YAML-based Ansible playbook file.
- name: Test Autofirstboot Properties Provider
gather_facts: false
hosts: Controllers
connection: "local"
run_once: true
tasks:
- name: Provide Autofirstboot Properties to Cluster
arista.dmf.provisioner:
config_json: "<Pave-Repave-config.json>"
timeout: 3600
log_dir: "logs"
The configuration playbook obtains the JSON-based configuration file from the server executing the playbook. The JSON file contains specific sections for each DMF appliance that is being installed/re-installed. The following is an example:
Note: The ansible script uses the initial-config-password to access the DMF appliance via SSH for the first time.
Ansible pushes initial configuration data to the appliance via SSH using the default credential configured in the JSON file in the steps above.
The appliance completes the initial configuration (first-boot).
The appliance confirms to Ansible that the initial configuration is complete.
The following is an example run of the Ansible playbook:
ansible-playbook sample_playbook.yml --limit="10.240.156.82,10.240.156.84" -v
[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the controller starting
with Ansible 2.12. Current version: 2.7.18 (default, Jul 1 2022, 12:27:04) [GCC 9.4.0].
This feature will be removed from ansible-core in version 2.12. Deprecation warnings can be
disabled by setting deprecation_warnings=False in ansible.cfg.
/root/.cache/pypoetry/virtualenvs/arista-dmf-vQFV2Q4_-py2.7/lib/python2.7/site-packages/
ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer
supported by the Python core team. Support for it is now deprecated in cryptography, and
will be removed in the next release.
from cryptography.exceptions import InvalidSignature
Using /etc/ansible/ansible.cfg as config file
PLAY [Test Autofirstboot Properties Provider]
*************TASK [Provide Autofirstboot Properties to Cluster]
*************Targeting all hosts specified in config [u'10.240.156.84', u'10.240.156.82']
[10.240.156.82 BMF (active)]: Waiting for SSH connection.
[10.240.156.84 BMF (standby)]: Waiting for SSH connection.
[10.240.156.84 BMF (standby)]: Established SSH connection.
[WARNING]: It is NOT recommended to disable SSL verification. Please upload a valid SSL
certificate to the https://10.240.156.84:8443 API server.
[10.240.156.82 BMF (active)]: Established SSH connection.
[WARNING]: It is NOT recommended to disable SSL verification. Please upload a valid SSL
certificate to the https://10.240.156.82:8443 API server.
[WARNING]: [10.240.156.82 BMF (active)]: Detected BIOS firmware. The BIOS PXE boot trigger has
been found to be unreliable. Upgrade to UEFI is recommended.
[WARNING]: [10.240.156.84 BMF (standby)]: Appliance already configured. Attempting PXE boot to
repave.
[WARNING]: [10.240.156.82 BMF (active)]: Appliance already configured. Attempting PXE boot to
repave.
[10.240.156.84 BMF (standby)]: Expecting appliance to reboot with IP = 10.240.156.84.
[10.240.156.82 BMF (active)]: Expecting appliance to reboot with IP = 10.240.156.82.
[10.240.156.84 BMF (standby)]: Verified repave succeeded.
[10.240.156.84 BMF (standby)]: Waiting for active to be configured.
[10.240.156.82 BMF (active)]: Verified repave succeeded.
[10.240.156.82 BMF (active)]: Successfully provided config to appliance.
[10.240.156.82 BMF (active)]: Waiting for appliance to apply config. Expecting final IP = 10.
240.156.82.
[10.240.156.82 BMF (active)]: Successfully retrieved firstboot logs.
[10.240.156.82 BMF (active)]: Config applied.
[10.240.156.82 BMF (active)]: Attempting to reset recovery password.
[10.240.156.82 BMF (active)]: Successfully reset recovery password.
[10.240.156.84 BMF (standby)]: Active configured. Proceeding with configuration.
[10.240.156.84 BMF (standby)]: Successfully provided config to appliance.
[10.240.156.84 BMF (standby)]: Waiting for appliance to apply config. Expecting final IP = 10.
240.156.84.
[10.240.156.84 BMF (standby)]: Successfully retrieved firstboot logs.
[10.240.156.84 BMF (standby)]: Config applied.
[10.240.156.84 BMF (standby)]: Attempting to reset recovery password.
[10.240.156.84 BMF (standby)]: Successfully reset recovery password.
changed: [10.240.156.82] => {"changed": true, "success": true, "summary": {"10.240.156.82": {
"appliance-type": "BMF", "expected-dhcp-ip": "10.240.156.82", "role": "active"}, "10.240.156.
84": {"appliance-type": "BMF", "expected-dhcp-ip": "10.240.156.84", "role": "standby"}}}
PLAY RECAP
******************56.82 : ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Assumption and Trust Model
PXE boot is fundamentally incompatible with zero-trust environments, and as a result, assumptions are made to establish the trust required to authenticate the appliances. This section provides a summary of the assumptions that underpin the security of this design.
The management network is trusted, such that:
DHCP is secured.
The DHCP server (or DHCP relaying) is secure.
Rogue DHCP packets are dropped/blocked.
Impersonation by MAC or IP spoofing is not possible due to either:
Assumption: Machines on the same L2 network are trusted not to impersonate other machines by presenting false identities (MAC addresses or IP addresses).
Guarantee: Machines in the same L2 network cannot impersonate other machines by presenting themselves with false identities as enforced by network admins who may, for example:
Pin a MAC address to a specific switch and switch port.
Pin an IP address to a MAC address statically (i.e., static ARP entry).
Routers between the PXE TFTP/HTTP server and the target machine are secure/trusted to forward packets to the rightful owners (e.g., having correct routing tables and MAC address tables).
The PXE (TFTP/NFS) server is secure and cannot be compromised, resulting in an attacker providing a malicious image.
This chapter describes the installation, initial configuration, and upgrade of the DMF Recorder Node.
frame-all
Overview
The DANZ Monitoring Fabric (DMF) Recorder Node (RN) is a traffic recording appliance with Arista Networks software running on Dell, Inc. servers.
The RN records packets from the network to disk and retrieves specific packets from disk quickly, efficiently, and at scale. It integrates with DMF for a single-pane-of-glass solution. A single DMF Controller can manage multiple RNs, delivering packets for recording through out-of-band policies. The Controller also provides centralized APIs for interacting with RNs to perform packet queries.
A DMF out-of-band policy directs the recording of matching packets to one or more RNs. The out-of-band policy defines the switch and port to attach the RN to the fabric. The policy treats these as “dynamic” delivery interfaces identified by unique names. The DMF Controller also provides commands for viewing errors, warnings, statistics, and the status of connected RNs.
The RN provides an OpenFlow agent that collects statistics and health information from the Controller. The OpenFlow agent also allows the Controller to configure the RN, eliminating the need to administer any RN directly during normal operation separately. To the DMF Controller, the OpenFlow agent causes the RN to appear as a special type of switch. Use the REST API to query the RN directly.
The DMF Recorder Node is based on the Dell server hardware and is available with the following interfaces:
Two management interfaces (10/100/1000 Mb/s)
One serial interface (DB-9)
One VGA interface
Two USB ports
One dedicated iDRAC port
Note: Arista recommends using an iDRAC connection to the DMF Controller, DMF Service Node, Arista Analytics Node, and DMF Recorder Node appliances. This connection helps with troubleshooting of issues. For more details, refer to the chapter on Using iDRAC later in this guide.
The following details the RN's storage capacity and the bandwidth provided by the data interfaces:
192 TB packet storage capacity
2 x 25 GbE SFP ports
2 x 10 GbE copper ports
The following figure illustrates the bezel on the larger (HWA) DMF Recorder Node.
Figure 1. DMF Recorder Node (HWA) Front Panel
1
System identification button/indicator
4
LCD panel
2
Recorder Node Security Bezel
5
Power-on indicator / Power button
3
LCD menu buttons
6
USB ports
The following figure illustrates the front panel of the DMF Recorder Node.
Figure 2. DMF Recorder Node (HWA) Front Panel
1
Information Tag
2
Video connector
3
Micro USB (not supported)
4
Hard drives
The following figure illustrates the rear panel of the DMF Recorder Node.
Press F11 to select the Boot Manager to allow booting from USB.
Figure 4. System Boot Manager Screen
Select One-shot BIOS Boot Menu.
The Boot Manager screen is displayed in the following figure.
Figure 5. Boot Manager Main Menu
Select the USB drive.
Figure 6. Boot Menu
Respond to the system prompt to login in using the admin account:
recorder-node login: admin
(Press Control-C at any time to cancel and start over)
This product is governed by an End User License Agreement (EULA).
You must accept this EULA to continue using this product.
You can view this EULA from our website at:
https://www.arista.com/en/eula
Do you accept the EULA for this product? (Yes/No) [Yes] >
Type Yes to accept the EULA, which is required to use the product. To view the EULA, type View, or refer to https://www.arista.com/en/eula.
The system displays the following messages.
Running system pre-check
Finished system pre-check
Starting first-time setup
Configure the recovery password.
Emergency recovery user password >
Emergency recovery user password (retype to confirm) >
Hostname > dmf-pr-740
Configure IP addresses for the management network and DNS servers.
[1] IPv4 only
[2] IPv6 only
[3] IPv4 and IPv6
> 1
IPv4 address [0.0.0.0/0] > 10.9.32.21/24
IPv4 gateway (Optional) > 10.9.32.1
DNS server 1 (Optional) > 10.3.0.4
DNS server 2 (Optional) >
DNS search domain (Optional) > qa.arista.com
Administrator password >
Administrator password (retype to confirm) >
Controller address if deployment mode is preconfigured (L3 ZTN) (Optional) > 10.111.35.101
If the RN is connected to the DMF Controller by a Layer 3 device (such as a router) in preconfigured (L3 ZTN) mode, enter the active DMF Controller's IP address.
Configure the administrator password.
Administrator password >
Administrator password (retype to confirm) >
Configure the NTP servers.
-----------
Default NTP servers:
- 0.bigswitch.pool.ntp.org
- 1.bigswitch.pool.ntp.org
- 2.bigswitch.pool.ntp.org
- 3.bigswitch.pool.ntp.org
NTP server options:
[1] Use default NTP servers
[2] Use custom NTP servers
[1] > 1
Confirm the settings.
Please choose an option:
[ 1] Apply settings
[ 2] Reset and start over
[ 3] Update Recovery Password (*****)
[ 4] Update Hostname (dmf-pr-740)
[ 5] Update IP Option (IPv4 only)
[ 6] Update IPv4 Address (10.9.32.21/24)
[ 7] Update IPv4 Gateway (10.9.32.1)
[ 8] Update DNS Server 1 (10.3.0.4)
[ 9] Update DNS Server 2 (<none>)
[10] Update DNS Search Domain (qa.arista.com)
[11] Update Admin Password (*****)
[12] Update NTP Option (Use default NTP servers)
[1] >
The system displays the following messages.
[Stage 1] Initializing system
[Stage 2] Configuring local node
Waiting for network configuration
IP address on bond0 is 10.9.32.21
Generating cryptographic keys
[Stage 3] Configuring system time
Initializing the system time by polling the NTP servers:
0.bigswitch.pool.ntp.org
1.bigswitch.pool.ntp.org
2.bigswitch.pool.ntp.org
3.bigswitch.pool.ntp.org
[Stage 4] Configuring cluster
Cluster is already configured
First-time setup is complete!
Press Enter to complete the configuration.
Initial Configuration - GUI
After completing the installation, refer to the DANZ Monitoring Fabric User Guide to configure and operate the Recorder.
GUI Procedure
Complete the following steps to use the DANZ Monitoring Fabric (DMF) GUI to configure the Recorder Node (RN).
Select Monitoring > Recorder Nodes from the main menu and click the Provision control (+) icon.
Figure 7. Add Recorder Node
Enter the required details by specifying a Name and identifying the MAC address of the RN appliance NIC connected to DMF.
Tip: Choose the MAC address from the selection list if it has been discovered.
Figure 8. Provision Recorder Node
Click Save.
Click the Provision control (+) at the top of the Interfaces section and enter the required information.
Configure the DMF policy to Identify the traffic to send to the RN.
controller-1(config-policy)# 1 match any
controller-1(config-policy)# filter-interface sw1-fil1
The following example forwards all traffic received in the monitoring fabric on filter-interface sw1-fil1 to the RN interface.
recorder-node pr-intf-1
description 'Delivery point for recorder node'
recorder-node-interface switch 00:00:70:72:cf:c7:cd:7d ethernet37
policy pkt-rec
action forward
filter-interface sw1-fil1
use-recorder-node pr-intf-1
1 match any
Changing the Recorder Node Default Configuration
Configuration settings are automatically downloaded to the Recorder Node (RN) from the DANZ Monitoring Fabric (DMF) Controller, eliminating the need for box-by-box configuration. DMF supports overriding the default configuration for an RN from the config-recorder-node submode for any RN.
Note: Currently, these options are available only from the CLI and not in the DMF GUI.
To change the CLI mode to config-recorder-node, enter the following command from config mode on the active DMF Controller.
Replace instance with the alias for the Recorder Node. This alias is associated with the MAC hardware address using the mac command.
Use any of the following commands from config-recorder-node submode to override the default configuration for the associated Recorder node.
banner: Set recorder-node pre-login banner message
mac: Configure MAC address for recorder-node name
Additionally, the following configurations can be overridden to use values specific to the recorder node or used in merge mode along with the configuration inherited from the Controller.
ntp: Configure packet-recorder to override default timezone and NTP parameters
snmp-server: Configure packet-recorder SNMP parameters and traps
logging: Enable packet-recorder logging to the Controller
tacacs: Set TACACS defaults, server IP address(es), timeouts and keys
To configure the recorder node to override the configuration inherited from the Controller, execute the following commands at the config-recorder-node submode:
ntp override-global: Override global time config with packet-recorder time config
snmp-server override-global: Override global SNMP config with packet-recorder SNMP config
snmp-server trap override-global: Override global SNMP trap config with packet-recorder SNMP trap config
logging override-global: Override global logging config with packet-recorder logging config
tacacs override-global: Override global TACACS+ config with packet-recorder TACACS+ config
To configure the recorder node to work in a merge mode by merging its specific configuration with that of the Controller, execute the following commands at the config-recorder-node submode:
ntp merge-global: Merge global time config with packet-recorder time config
snmp-server merge-global: Merge global SNMP config with packet-recorder SNMP config
snmp-server trap merge-global: Merge global SNMP trap config with packet-recorder SNMP trap config
logging merge-global: Merge global logging config with -packet-recorder logging config
The TACACS+ configuration does not provide a command usable with the merge option: it can be inherited from the Controller or overridden to use only the recorder node-specific configuration.
This chapter describes how to install the DANZ Monitoring Fabric DMF Service Node.
Overview
The DANZ Monitoring Fabric (DMF) Service Node provides advanced packet matching and modification capabilities for monitored traffic. The DMF Service Node is an optional component in the DANZ Monitoring Fabric, providing advanced features. The DMF Service Node offers the following services:
Deduplication
Header stripping
IPFIX generation
Packet masking
NetFlow
Pattern dropping
Pattern matching
Packet slicing
Timestamping
UDP replication
Figure 1. DMF Service Node
After the basic installation, the DMF Controller automatically detects each connected DMF Service Node, and the Controller starts managing the DMF Service Node Appliance along with the connected fabric switches.
For information about configuring and using the DMF Service Node, refer to the DANZ Monitoring Fabric 8.4 User Guide.
Connecting the Service Node
Connect the DMF Service Node to a core interface on a DMF switch, which provides access to filter interfaces, where traffic is received for the DMF Service Node, and to delivery interfaces, where traffic is delivered after it is processed. The service node can be connected in two ways:
Using the management interface (1 GbE) connected to the management switch. Using the management interface limits throughput to 1 GbE and shares the bandwidth with management traffic to the DMF Controller.
Using the 10 GbE Service Node interfaces (SNI) connected to the DMF switches. Connecting to this interface supports up to 10 GbE throughput.
Note: Arista recommends having an iDRAC connection to the DMF Controller, DMF Service Node, Arista Analytics Node, and DMF Recorder Node appliances. This connection helps in easy troubleshooting of issues. For more details, refer to the chapter on Using iDRAC later in this guide.
The figure below shows the interfaces provided on the 4-port DMF Service Node Appliance.
Figure 2. DMF Service Node (4-Port Appliance)
1
Service Interfaces (10G)
2
Service Node Management Port 1 (1000 Mb/s) - Connect to the management switch.
3
Serial Connector
Figure 3. DMF Service Node BL (16-Port Appliance)
1
Service interfaces SNI13
10
Service interfaces SNI10
2
Service interfaces SNI15
11
Service interfaces SNI9
3
Service interfaces SNI16
12
Service interfaces SNI5
4
Service interfaces SNI14
13
Service interfaces SNI4
5
Service interfaces SNI8
14
Service interfaces SNI3
6
Service interfaces SNI12
15
Service interfaces SNI2
7
Service interfaces SNI11
16
Service interfaces SNI1
8
Service interfaces SNI17
17
Ethernet Connector 1 Service Node Management Port 1 (10/100/1000 Mb/s)
9
Service interfaces SNI6
18
Serial Connector
Service Node Setup and Initial Configuration
This section describes how to perform the initial setup and configuration on a new DANZ Monitoring Fabric (DMF) Service Node appliance.
Note: Disable hyperthreading on the hardware appliance before installing the Service Node software to avoid performance issues. When disabling hyperthreading after installing the Service Node software, performance will be affected, and reinstalling the software will be required to resolve the issue.
Complete the following steps to run the first boot setup, which performs the initial configuration required for a new DMF Service Node.
Procedure
Rack the DMF Service Node Appliance.
The appliance interfaces are on the back of the appliance, where the power cord connects. These include the following:
Four management interfaces (10/100/1000 Mb/s): You can connect either of the two lower left interfaces (Ethernet 1 or Ethernet 2) to the network management switch.
One serial interface (db9).
Four 10-GbE SFP ports on the R640 server and 16 10-GbE ports on the R740 server. Connect these ports to a DMF switch.
Turn on the DMF Service Node server appliance.
Log in via the serial port using the admin account name. The baud rate is 115200. When using a terminal server to connect, ensure the baud rate on the terminal server is 115200.
When the first boot process begins, accept the End User License Agreement (EULA).
This product is governed by an End User License Agreement (EULA). You must accept this
EULA to continue using this product.
You can view this EULA by typing 'View', or from our website at. https://www.arista.com/en/eula
Do you accept the EULA for this product? (Yes/No/View) [Yes] > yes Running system pre-check
Finished system pre-check
Starting first-time setup
Complete the local node configuration according to the requirements of your network environment.
The following is only an example. Change for your specific deployment.
Local Node Configuration
------------------------
Emergency recovery user password >
Emergency recovery user password (retype to confirm) >
Hostname > DMF-Service-Node
Management network options:
[1] IPv4 only
[2] IPv6 only
[3] IPv4 and IPv6
>1
IPv4 address [0.0.0.0/0] > 10.8.39.200/18
IPv4 gateway (Optional) > 10.8.0.1
DNS server 1 (Optional) > 10.3.0.4
DNS server 2 (Optional) > 10.1.5.200
DNS search domain (Optional) > qa.arista.com
Administrator password >
Administrator password (retype to confirm) >
Controller address if deployment mode is preconfigured (L3 ZTN) (Optional) > 10.106.6.4
If the DMF Service Node is connected to the DMF Controller by a Layer 3 device (such as a router) in preconfigured (L3 ZTN) mode, enter the active DMF Controller's IP address.
Note: Starting with DMF Release 7.1.0, the DMF Service Node can be installed using Zero Touch Fabric (ZTF) even if it is in a different subnet than the DMF Controller.
Identify the Network Time Protocol (NTP) servers.
System Time
-----------
Default NTP servers:
- 0.bigswitch.pool.ntp.org
- 1.bigswitch.pool.ntp.org
- 2.bigswitch.pool.ntp.org
- 3.bigswitch.pool.ntp.org
NTP server options:
[1] Use default NTP servers
[2] Use custom NTP servers
[1] > 1
When prompted, type 1 to apply the selected options, or type any number on the menu that is displayed to change the current setting.
Please choose an option:
[ 1] Apply settings
[ 2] Reset and start over
[ 3] Update Recovery Password (*****)
[ 4] Update Hostname (R740)
[ 5] Update IP Option (IPv4 only)
[ 6] Update IPv4 Address (10.106.6.7/23)
[ 7] Update IPv4 Gateway (10.106.6.1)
[ 8] Update DNS Server 1 (10.108.200.200)
[ 9] Update DNS Server 2 (10.100.5.200)
[10] Update DNS Search Domain (qa.arista.com)
[11] Update Admin Password (*****)
[12] Update Controller IP (10.106.6.4)
[13] Update NTP Option (Use default NTP servers)
[1] > 1
After first-time setup is complete, press Enter to continue.
[Stage 1] Initializing system
[Stage 2] Configuring local node
Waiting for network configuration IP address on bond0 is 10.8.39.200 Generating
cryptographic keys
[Stage 3] Configuring system time
Initializing the system time by polling the NTP servers:
0.bigswitch.pool.ntp.org
1.bigswitch.pool.ntp.org
2.bigswitch.pool.ntp.org
3.bigswitch.pool.ntp.org
[Stage 4] Configuring cluster Cluster configured successfully. Current node ID is 27521
All cluster nodes:
Node 27521: 10.8.39.200:6642
First-time setup is complete!
Press enter to continue >
DMF Service Node (dmf-8.0-service-node #1) Log in as 'admin' to configure
Connect one or more of the 10 GbE SFPs on the appliance hardware to a DMF out-of-band switch.
The DMF Controller automatically detects each connected DMF Service Node Appliance and integrates the service node into the monitoring fabric.
Note: The DMF Controller will not establish a connection to the DMF Service Node Application if none of the 10 GbE SFPs on the appliance hardware are connected to a DMF out-of-band switch.
Once connected to the DMF Controller, it can take several minutes for Service Node to update the software.
Figure 4. Service Node software update
Login to the DMF Controller.
Add the following service node configuration with the Service Node management interface's MAC address.
Note: Obtain the MAC address of the Service Node by logging into the service node via SSH and running the following command taking note of the bond0 MAC address:
SN-1(config)# show local-node interfaces bond0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Interfaces ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Interface Master Hardware address Permanent hardware address Operstate Carrier Bond mode Bond role
---------|------|------------------------|--------------------------|---------|-------|-------------|---------|
bond0 78:ac:44:05:65:b8 (Dell) up up active-backup
~~~~~~~~~~ Addresses of Interfaces ~~~~~~~~~~
# Interface Ip cidr
-|---------|---------------------------------|
1 bond0 10.240.130.26/25
2 bond0 fe80:0:0:0:7aac:44ff:fe05:65b8/64
Verify that the DMF Service Node is connected by entering the following command:
controller-1(config-service-node)# show managed-service-device
Enter this command in any CLI mode.
Creating Support Bundle on Service Node
The support bundle for the DANZ Monitoring Fabric (DMF) Service Node should be created from the DMF Controller (Creating a Support Bundle). But, if the DMF Service Node loses connectivity to the DMF Controller, a support bundle can be created by logging into the DMF Service Node.
The DMF Service Node CLI provides commands to automate the collecting, archiving, and uploading of critical data.
The following are the commands to configure Support Bundle auto-upload:
LG-SN(config)# service
LG-SN(config-service)# support auto-upload
<cr>
LG-SN(config-service)# support auto-upload
Enabled diagnostic data bundle upload
Use "diagnose upload support" to verify upload server connectivity
LG-SN(config-service)#
To check if auto-upload is enabled or not:
LG-SN(config-service)# show run service
! service
service
support auto-upload
LG-SN(config-service)#
The following is the command to generate the Support Bundle. After generating the support bundle, it uploads automatically. Please provide the support bundle ID to support personnel.
LG-SN(config-service)# support
Generating diagnostic data bundle for technical support. This may take several minutes...
Support Bundle ID: SMFUG-BS5S2
Local cli collection completed after 32.9s. Collected 33 commands (0.14 MB)
Local rest collection completed after 0.0s. Collected 3 endpoints (0.17 MB)
Local bash collection completed after 93.3s. Collected 133 commands (6.73 MB)
Local file collection completed after 8.4s. Collected 42 paths (1851.13 MB)
Collection completed. Signing and compressing bundle...
Support bundle created successfully
00:03:16: Completed
Generated Support Bundle Information:
Name : anet-support--LG-SN--2022-04-13--07-46-01Z--SMFUG-BS5S2.tar.gz
Size : 490MB
File System Path : /var/lib/floodlight/support/anet-support--LG-SN--2022-04-13--07-46-01Z--
SMFUG-BS5S2.tar.gz
Url : https://10.240.130.8:8443/api/v1/support/anet-support--LG-SN--2022-04-
13--07-46-01Z--SMFUG-BS5S2.tar.gz
Bundle id : SMFUG-BS5S2
Auto-uploading support anet-support--LG-SN--2022-04-13--07-46-01Z--SMFUG-BS5S2.tar.gz
Transfer complete, finalizing upload
Please provide the bundle ID SMFUG-BS5S2 to your support representative.
00:01:03: Completed
LG-SN(config-service)#
The show support command shows the status of the automatic upload.
LG-SN(config-service)# show support
#Bundle Bundle idSize Last modified Upload status
- |------------------------------------------------------------------- |----------- |----- |------------------------------ |---------------- |
1anet-support--LG-SN--2022-04-13--07-46-01Z--SMFUG-BS5S2.tar.gz SMFUG-BS5S2490MB2022-04-13 07:49:20.157000 UTCupload-completed
2anet-support--LG-SN--2022-04-13--07-19-08Z--SI51T-BVJJB.tar.gz SI51T-BVJJB488MB2022-04-13 07:22:44.927000 UTC
3anet-support-component--LG-SN--2021-05-19--22-47-17Z_0kc7zxw.tar.gz 462MB2021-05-19 22:47:17.452000 UTC
LG-SN(config-service)#
Tip: Use the diagnose upload support command to check the reachability and health of the server before uploading the support bundle.
LG-SN(config-service)# diagnose upload support
Upload server version: diagus-master-76
Upload diagnostics completed successfully
00:00:04: Completed
Check : Resolving upload server hostname
Outcome : ok
Check : Communicating with upload server diagnostics endpoint
Outcome : ok
Check : Upload server healthcheck status
Outcome : ok
Check : Upload server trusts authority key
Outcome : ok
Check : Signature verification test
Outcome : ok
Check : Resolving objectstore-accelerate hostname
Outcome : ok
Check : Resolving objectstore-direct hostname
Outcome : ok
Check : Communicating with objectstore-accelerate
Outcome : ok
Check : Communicating with objectstore-direct
Outcome : ok
LG-SN(config-service)#
Upgrading Service Node Software From Release 7.x.x
Upgrading the DANZ Monitoring Fabric (DMF) Service Node, deployed in L2ZTN from DMF-7.0.0 to a later version, will be automatically completed through a zero-touch upgrade if you upgrade a DMF Release 7.0.x Controller.
Upgrade of DMF Service Node deployed in L3ZTN from DMF- 7.1.0 to a later version will be automatically completed through zero-touch when you upgrade a DMF Release 7.1.x Controller.
To verify that the Service Node is ready for the zero-touch upgrade, enter the following command from the CLI prompt on the active DMF Controller.
This chapter describes how to upgrade the DANZ Monitoring Fabric (DMF) Controller and fabric switches after initial installation.
Upgrading the Controller
The default serial console baud rate on DMF 6.1 and later hardware appliances is 115200. Arista recommends against using the serial interface to perform an upgrade. However, when upgrading an existing Controller appliance with a serial interface set to 9600 baud, change the terminal setting to 115200 after upgrading to DMF 6.1.0 or later.
Note: The upgrade process checks for supported versions when upgrading. Refer to the most recent DANZ Monitoring Fabric 8.5.0 Release Notes for a list of supported versions for upgrade.
The DANZ Monitoring Fabric (DMF) Controller platform supports two partitions for Controller software images. The active partition contains the active image currently running on the Controller. The alternate partition can be updated without interrupting service. The image used for booting the Controller is called the boot image. In addition, the Controller has an image repository in its local file system where you can copy upgrade images. The upgrade image is verified for integrity before it is copied and rejected if it fails the checksum test.
Note: Copying an upgrade bundle to the Controller overwrites any older image currently in the image repository. When downloading a bundle identical to the image repository version, the operation fails when calculating the checksum after copying is complete.
After copying the upgrade image to the image repository, use the upgrade
stage command to copy the upgrade image to the alternate partition and prepare the Controller for the upgrade.
The boot image remains in the active partition after copying the ISO image file to the alternate partition. After entering the upgrade launch command, the upgrade image is changed to the boot image and the Controller reboots.
Note: The upgrade process is not hitless.
The upgrade launch command copies the active state of the Controller to the alternate partition and reboots the Controller using the upgrade image. After completing the upgrade process, the alternate and active partitions are reversed, the Controller is running with the upgrade image, and the older image remains available in the alternate partition until a new image is copied into it, which will overwrite it.
As long as the original image remains in the alternate partition, use the boot partition alternate command to restore the Controller to its previous software and configuration.
Log files are written in a separate partition, available from either of the two images by entering the show logging controller command.
Upgrade Procedure Summary
Complete the following steps to upgrade the Controllers. The switch upgrade process, using ZTF, happens automatically (see “DMF Switch Automatic Upgrade section for details)
Note: Only the admin user can perform an upgrade or enter the show image command.
Procedure
Log in to the active Controller using its individual IP address or the virtual IP address for the cluster.
From the active Controller, enter the upgrade stage command and respond to the prompts as required (see the Staging the Upgrade section for details).
From the active Controller, enter the upgrade launch command and respond to the prompts as required (see the Launching the Upgrade section for details).
Note: Do not attempt to launch or stage another upgrade process until the current process is either completed or times out.
Wait for the process to complete and verify the upgrade before making any configuration changes.
The following options are available for use with the upgrade command.
cluster: with the cluster option, the upgrade command is executed cluster-wide. The user must log in to the active Controller to run the cluster option.
launch: Complete the upgrade process by rebooting the Controller from the alternate partition and transferring state and configuration information from the Controller to the upgraded Controller and its running-config. This keyword manages the transition from the current version to the next version, which may include rebooting all Controllers in the cluster, rebooting.
Use the following options with the launch keyword as required:
support-bundle: Generate a support bundle for use by Arista tech support.
switch-timeout: Optionally, specify the number of seconds to wait before terminating the command to a switch during the upgrade.
pre-launch-check: Identifies the status of the Controller regarding readiness for upgrade.
stage: Prepares the platform for the upgrade ahead of the actual upgrade process by copying the upgrade image to the alternate partition on the Controller.
Copying the ISO Image File to the Controller
The ISO image file is a software image that includes the files required to complete the upgrade of the Controller. It also contains the files for installing or upgrading the Switch Light OS image on switches. The primary component of the upgrade image is a new root file system (rootfs) for the Controller.
When copying the upgrade image, if there is not enough file space on the local file system, the system prompts to create space on the local file system. During the image copy process, if the integrity of the image cannot be verified, the system displays a message, and the copy fails. After copying the image, a warning message appears if the image is not a compatible upgrade image for the existing application, or if the image is not a newer version.
Note: Starting from DMF release 8.4, due to an increased image size, when trying to copy a new image you will likely encounter the following error:
Error: Invalid Use: No space left, use "delete image" command to free space
It is therefore required to remove all images from the image partition, including the image for the currently running version, to free up space.
CONTROLLER-1# show cluster image
# Checksum Cluster NodesProductVersion Build
-|--------|--------------------------|----------------------|-------|-----|
1 d301aCONTROLLER-1, CONTROLLER-2 DANZ Monitoring Fabric 8.5.0 33
CONTROLLER-1#
CONTROLLER-1# delete cluster image all
all: delete: confirm ("y" or "yes" to continue): yes
CONTROLLER-1#
Note that you can also delete images on the individual Controllers like so:
CONTROLLER-1# show image
CONTROLLER-1# delete image all
Even though you have deleted all the images from the /images repository, you can still roll back to the current version after the next upgrade operation, as the current image is still present in the boot partition.
To use the Controller CLI to copy the ISO image file to the Controller Image repository from an external server, use the copy command, which has the following syntax:
controller-1# copy <source> <destination>
Replace source with a location accessible from the Controller node using one of the following protocols:
scp://<user>@<host>:path: Remote server filename, path, and user name with access permissions to the file. The remote system prompts for a password if required.
http://: HTTP URL including path and filename.
https://: HTTPS URL including path and filename
Replace destination with image://cluster. This will download the image from the remote server to all the nodes in the cluster. For example, the following command copies the file DMF-<X.X.X>-Controller-Appliance-<date>.iso from myscpserver.example.com to the Controller alternate partition for both active and standby Controller:
controller-1# # copy scp://This email address is being protected from spambots. You need JavaScript enabled to view it.:*DMF-<X.X.X>-Controller-Appliance-<date>.
iso* image://cluster
Use the show cluster image command to verify that the ISO image file has successfully been copied to both the active and standby Controller, as shown in the following example:
controller-1# show cluster image
Staging the Upgrade
The upgrade cluster stage command applies the specified upgrade image into the alternate partition for both active and standby Controllers. This command prepares the platform for an upgrade by populating the alternate partition with the contents of the new image. This is a separate step from the launch step, where the upgraded image becomes the active image to prepare the platform ahead of the upgrade process.
To view the upgrade images currently available, use the show upgrade image or show cluster image commands.
Use the upgrade cluster stage command to prepare the Controllers for the upgrade. This action copies the running-config to a safe location and marks the alternate partition as the boot partition.
The system responds as shown in the following example:
controller-1# # upgrade cluster stage
Upgrade stage: alternate partition will be overwritten
proceed ("yes" or "y" to continue):y
Note: Do not attempt to launch or stage another upgrade process until the current process is either completed or times out.
At this point, enter y to continue the staging process. The system continues the process and displays the following prompts:
Upgrade stage: copying image into alternate partition
Upgrade stage: checking integrity of new partition
Upgrade stage: New Partition Ready
Upgrade stage: Upgrade Staged
To verify that the system is ready for upgrade, enter the show boot partition command, as in the following example:
controller-1# #show boot partition
This command lists the available partitions and the information about the Controller versions installed on each. In this example, the original image remains in active state and is still the boot image, meaning if a reboot occurs now, the original image is used to reboot. The upgrade image will not be used to boot until it has been changed to the boot image, which is one of the effects of the upgrade cluster launch command.
To display the current status of the upgrade process, you can use the show upgrade progress command, as in the following example:
controller-1# # show upgrade progress
Note: Upgrade requires a successfully staged image partition. If the upgrade stage is interrupted, it is necessary to stage the image again to launch the upgrade.
Launching the Upgrade
The upgrade cluster launch command reboots the system using the upgrade image in the alternate partition and copies the current state from the previous Controller. The system collects information from the existing cluster, for example, the current running- config and saves this state in the staged partition, so it is available for the new cluster.
Once the running-config is applied, the existing operational state of the existing cluster is requested and then applied to the new cluster. The new cluster requests the switches from the old cluster, adjusts its local configuration to describe this new ownership, and reboots the switches.
The upgrade cluster launch command manages the transition from the current version to the next for both active and standby Controllers. This includes various steps, including rebooting all the Controllers in the cluster, rebooting all the switches, and upgrading the switches. After the standby node is upgraded, the roles of active and standby are reversed, so the upgraded node can assume control while the other node is upgraded and reboots.
Note: Do not attempt to launch or stage another upgrade process until the current process is either completed or times out.
Log into the active Controller via the Controller management IP or virtual IP and enter the upgrade cluster launch command:
controller-1# upgrade cluster launch
The system prompts to proceed with the reboot. Enter yes. The standby Controller reboots and displays the following messages while the active Controller waits for the standby Controller to boot up:
Upgrade Launched, Rebooting
Broadcast message from root@controller (unknown) at 19:40 ...The system is going down for
reboot
NOW!User initiated reboot
While rebooting, the SSH terminal session is terminated. Reconnect after the reboot is complete.
DMF Switch Automatic Upgrade
When the fabric switch reboots, it compares its current software image, manifest, and startup-config to the information in the switch manifest that the Controller sends after the switch reboots. The switch optimizes the process by caching its last known good copies of the software image and the startup-config in its local flash memory.
The switch automatically starts the upgrade process when it reboots if the ZTF server has a software image or startup-config with a different checksum than it currently has. This check is performed every time the switch restarts.
Verifying the Upgrade
After completing the upgrade process, verify that both Controllers and the fabric switches have been upgraded to the correct DANZ Monitoring Fabric and Switch Light OS versions.
To verify the upgrade of the Controllers, log in to each Controller and enter the show version command, as shown in the following example:
controller-1># show version
To verify the upgrade of the switches, enter the show switch all command, as shown in the following example:
controller-1># show switch all desc
This command displays the current switch version.
Verify Persistent IPAM Assigned IP Addresses after Upgrade
IPAM-assigned IP addresses for switches are persistent across DMF upgrades.
Configuration
No additional configuration changes are required to keep IPAM-assigned IP address configuration persistent across cluster upgrades. However, this assumes that IPAM has been previously configured.
When deciding to rollback or downgrade the Controller software after an upgrade, note that the rollback or downgrade is not hitless and will take several minutes to complete. After both Controllers are up and have joined the cluster, check each switch version and reboot each switch as needed to ensure all the switches have an image compatible with the Controller version.
To restore the system to the previous image, complete the following steps:
Procedure
On both the active and standby Controllers, enter the show boot partition command to ensure that the previous image remains in the Alternate partition.
controller-1># show boot partition
Reboot the active Controller node from the alternate partition by entering the boot command from the CLI prompt of the active Controller.
controller-1># boot partition alternate
Next Reboot will boot into partition 1 (/dev/vda2)
boot partition: reboot? ("y" or "yes" to continue): yes
Answer yes when prompted.
Reboot the standby Controller node from the alternate partition by entering the boot command from the CLI prompt of the standby Controller.
standby controller-2># boot partition alternate
Next Reboot will boot into partition 1 (/dev/vda2)
boot partition: reboot? ("y" or "yes" to continue): yes
Answer yes when prompted.
After both Controller nodes have restarted, reboot all switches by entering the system reboot switch command on the active Controller.
controller-1># system reboot switch all
system switch reboot all: connected switches:
00:00:70:72:cf:c7:06:bf
00:00:70:72:cf:c7:c5:f9
00:00:70:72:cf:ae:b7:38
00:00:70:72:cf:c8:f9:25
00:00:70:72:cf:c7:00:ad
00:00:70:72:cf:c7:00:63
reboot may cause service interruption
system switch reboot all ("y" or "yes" to continue): yes
Answer ``yes`` when prompted.
Wait for all the switches to reconnect.
controller-1># show switch
Verify that the standby Controller has rejoined the cluster with the reverted active Controller by entering the show controller command from both nodes.
controller-1># show controller
Any connected DMF Service Node or DMF Recorder Node must be upgraded after upgrading DMF fabric Controllers and switches.
Upgrade from DMF Release 7.1.0 to later versions uses Zero Touch Fabric (ZTF) and occurs automatically for connected nodes after the DMF Controller is upgraded.
Note: If the DMF Recorder Node is in a different Layer 2 segment than the DMF Controller, a fresh installation of the Recorder is required to upgrade to Release 7.1.0.
This chapter describes how to manage switches and interfaces after installing the monitoring fabric switches.
Connecting Directly to a Switch
To install the Switch Light OS individually on a switch in a different Layer 2 domain or troubleshoot the switch, use telnet or SSH to connect to the switch.
To allow SSH to a switch, if using ZTF for installing switches from the DANZ Monitoring Fabric (DMF) Controller in the same Layer 2 domain, configure an IPAM IP address pool. Alternatively, use the connect switch switch-name command to connect to the switch CLI.
controller-1(config)# connect switch DMF-FILTER-SWITCH-1
Warning: Permanently added the RSA host key for IP address 'fe80::3617:ebff:fef2:cfc4%em1' to the
list of known hosts.
Last login: Mon Sep 18 02:32:35 2017 from fe80::46a8:42ff:fe35:29f7%ma1
SwitchLight ZTN Manual Configuration. Type help or ? to list commands.
After connecting to the switch, enter the debug admin command at the displayed ztn-config prompt.
(ztn-config) debug admin
DMF-FILTER-SWITCH-1>
This command provides access to the switch CLI prompt for directly managing or troubleshooting the switch. The following commands are available in the ZTN console:
controller: adds, removes, sets, or clears the L3 ZTN Controller list
debug: Special command to access the full switch CLI
help: Displays CLI help
interface: sets the ma1 address parameters
reboot: restarts the switch
setup: performs interactive setup
show: displays the current settings
Manually Configuring Enhanced Hashing for Load Distribution
In some scenarios, it may be desirable to manually select the bytes in each packet that are used for load distribution among the members in a LAG.
Enter the hash-type enhanced command from the config-switch-lag-if submode to manually configure enhanced hashing.
Use the lag-enhanced-hash command to enter the config-switch-hash submode and use the hash command to identify the values to use for load distribution.
Note: For a list of the switch platforms that support enhanced hashing (including symmetric hashing), refer to the DANZ Monitoring Fabric Hardware Compatibility List.
Changes in hash configuration do not affect the LAG configuration, so there is no need to reconfigure LAGs after changing the hash type.
Configuring Enhanced Hashing
To configure enhanced hashing, use the lag-enhanced-hash command to enter the config-switch-hash submode. Use the hash command to identify the hash type and the specific fields for load distribution.
Enhanced hashing supports symmetric load balancing (enabled by default) for switch platforms that support this feature.
Note: DANZ Monitoring Fabric (DMF) supports symmetric hashing on specific switches for IP and Fibre Channel over Ethernet (FCoE) traffic. Symmetric hashing on MPLS traffic labels is not supported.
With symmetric load balancing, the link selected for distributing traffic in one direction is also used for traffic in the other direction. Arista Networks recommends enabling hashing on the source IP address and destination IP address for optimal symmetric behavior.
Note: In some scenarios, using Layer 4 protocol ports can improve load-balancing efficiency. However, these fields are not used by default because they cannot be used if packet fragmentation is likely to occur.
GTP Hashing
Generic Tunneling Protocol (GTP) hashing provides a more even distribution of GTP-encapsulated packets among the members of a port-group. When GTP hashing is enabled, DANZ Monitoring Fabric (DMF) includes the Tunnel endpoint identifier (TEID) value in the GTP packets in its hashing algorithm for outbound traffic. This applies only to GTP user data tunneling packets (udp port 2152). GTP control traffic (udp port 2123) is not affected.
To enable hashing with Generic Tunneling Protocol (GTP), use the hash
gtp command. This command sets enhanced hash parameters for distributing traffic on port-channel member ports for which enhanced hashing is enabled. The command syntax is as follows:
The GTP command specifies the packet fields to identify GTP traffic. When enabled, DMF uses the TEID in the GTP header for hashing GTP traffic instead of using L4 ports—Configure l4-dst-port and l4-src-port in hash ipv4 or hash ipv6 for proper operation.
Overriding the Default Switch Configuration
After completing the switch installation, further switch configuration, including software upgrades, is managed from the DANZ Monitoring Fabric (DMF) Controller.
CAUTION: Make all configuration changes related to fabric switches using the Controller CLI or the Controller GUI, which provides DMF-level configuration options in the config-switch submode for each switch. Do not log in to the switch to make changes directly using the switch CLI.
In general, the configuration options set on the DMF Controller are pushed to each connected switch, eliminating the need for box-by-box configuration. However, merging or overriding the default configuration pushed from the DMF Controller with switch-specific configuration for some parameters is possible.These parameters are as follows:
Clock
SNMP and SNMP Traps
Logging
TACACS
DMF supports two types of overriding mechanisms:
override-global: Only the switch-specific configuration is applied.
merge-global: The global config and switch-specific configuration are merged and then applied.
In the merge mode, the effective switch configuration is determined by the following rules:
Stand-alone values:If the key only exists in one of the configurations; take it as-is in the resultant configuration, otherwise:
If the key exists in both global and switch configurations: the value of the key from the switch-config takes precedence (over its value from the global-config).
Lists: If the list only exists in one of the configurations; take that list as-is in the result configuration, otherwise:
If it exists in both global and per-switch configuration, then merge with this rule.
If the global and switch-specific configuration has an entry with the same key, the switch-specific list entry completely replaces the entry from the global-config, otherwise:
All entries from the switch-specific configuration are appended to the global-config (with de-duplication). The configurations that occur as lists for the above overridable parameters are indicated below:
ntp
server <- list
time-zone
snmp-server
community <- list
contact
enable
host <- list
location
switch trap
user <- list
logging
controller
remote
remote server <- list
tacacs
server <- list
GUI Procedure
Select Fabric > Switches from the main menu.
The system displays the Switches page, which lists the switches connected to the DMF controller.
To override any of the default switch configuration settings, click the Menu control next to the switch and select Configure from the pull-down menu that appears.
Figure 1. Configure Switch (Page 1)
This dialog provides access to a series of dialogs used to override the default configuration that is pushed from the DMF Controller to the switch.
To advance to another page, click the numbered link for the page or click Next.
After making any changes required, click Submit.
CLI Procedure
To override the default configuration for a specific switch, enter the config-switch submode for the specific switch and use the commands available, viewable by entering the help command or by using tab completion.
controller-1# config
controller-1(config) switch DMF-FILTER-SWITCH-1
controller-1(config-switch) <Tab>
admin lag-interface sflow switch-group
banner logging show
tacacs
description mac shut down
tunnel-interface
interface ntp snmp
lag-enhanced-hash role snmp-server
controller-1 (config-switch)#
Use the shutdown command to shut down a switch from the Controller, in which case all the interfaces of the switch are put in admin down mode and the switch is black-holed.
Configuring Switch Interfaces
To use the GUI to configure switch interfaces, complete the following steps.
Procedure
Select Fabric > Interfaces from the GUI Main menu.
Figure 2. Fabric Interfaces Option
This page allows monitoring and configuring the interfaces on switches connected to the DANZ Monitoring Fabric (DMF) Controller. To display details about a specific interface, click the Expansion control to the left of the interface, and the entry expands to show any Tunnel Interfaces or Core Links currently using the interface.
To configure an interface, click the Menu control next to the interface and select Configure from the pull-down menu.
Figure 3. Configuring Interface Settings - Page 1
This dialog provides access to three pages.
Note: Page 3 is the DMF page configuration settings for interfaces to use in policies. For further information, refer to the DANZ Monitoring Fabric User Guide.
Page 1: The Port dialog provides the following options:
Admin Status: Enable or disable the switch administratively.
Enable Optics: Change the default to cause the optical laser to be left on after the port goes down.
MAC Loopback Mode: Returns traffic to the originating interface.
Force Link Up: This is useful to enable when only the transmit fiber is connected.
Description: Assign a description for the interface.
Tip:
Ideally, only apply a force link-up configuration on a delivery interface.
This configuration allows L1 on the port to stay up, even when the optical fiber cable is connected only in the TX direction.
This feature helps to black hole traffic if applied on links between switches.
When finished, click Save. To configure traffic options, click Next.
After clicking Next, the system displays the following page.
Use Page 2 of the Edit Interface dialog to explicitly enable or disable forward error correction or to restore the default.
CLI Procedure
To use the CLI to explicitly enable or disable forward error correction or restore the default, use the following commands from config-switch mode:
controller-1(config-switch)# interface ethernet10
controller-1(config-switch-if)# forward-error-correction
disable Force disable interface forward-error-correction
enable Force enable interface forward-error-correction
enable-fire-code Force enable interface fire-code forward-error-correction
enable-reed-solomon Force enable interface reed-solomon forward-error-correction
enable-reed-solomon544 Force enable interface reed-solomon544 forward-error-correction
controller-1(config-switch-if)# forward-error-correction enable
controller-1(config-switch-if)# show this
! switch
switch DMF-FILTER-SWITCH-1
!
interface ethernet10
autoneg disable
forward-error-correction enable
controller-1(config-switch-if)# forward-error-correction disable
controller-1(config-switch-if)# show this
! switch
switch DMF-FILTER-SWITCH-1
!
interface ethernet10
autoneg disable
forward-error-correction disable
controller-1(config-switch-if
Replace intf-port-list by the interface name or port list.
The following summarizes the effect of each forward error correction keyword option:
disabled – Disable if possible in the current port context.
enabled – Enable if possible in the current port context.
enable-fire-code – Request Fire-Code FEC (CL74) on port on port context.
enable-reed-solomon – Request Reed-Solomon FEC (CL91, CL108) on port context.
enable-reed-solomon544 – Force Reed-Solomon544 on port context.
Note: Switch platforms with the Tomahawk ASIC cannot support Reed-Solomon FEC (CL108) on 25G interfaces. FEC options supported on 25G interfaces are limited Fire-Code FEC or FEC Disabled. This limitation does not apply to 100G interfaces.
Autonegotiation
The following summarizes the effect of each option:
autoneg enabled – Enable if possible in the current port context.
autoneg disabled – Disable if possible in the current port context.
Autonegotiation can be enabled or disabled for the following interface configurations:
100 GbE DAC in 100 GbE mode
1G-BASE-SX
1G-BASE-LX
GUI Procedure
Use Page 2 of the Edit Interface dialog to enable or disable autonegotiation, or to restore the default.
CLI Procedure
To use the CLI to explicitly enable or disable autonegotiation, or to restore the default, enter the following command from config-switch mode for any fabric switch:
controller-1(config-switch)# interface ethernet10
controller-1 (config-switch-if)# autoneg enable
controller-1 (config-switch-if)# show this
! switch
switch DMF-FILTER-SWITCH-1
!
interface ethernet10 autoneg enable
controller-1 (config-switch-if)# autoneg disable
controller-1 (config-switch-if)# show this
! switch
switch DMF-FILTER-SWITCH-1
!
interface ethernet10
autoneg disable
controller-1 (config-switch-if)#
Replace intf-port-list by the interface name or port list.
Manually Setting the Interface Speed
To manually set the interface speed for an interface, from config-switch-if submode, enter the speed command, which has the following syntax.
DANZ Monitoring Fabric (DMF) supports using breakout (splitter) cables to split a single 40 GbE, 100 GbE, 200 GbE, and 400 GbE port into individual sub-interfaces.
For a list of supported switches, ports, and breakout cables, refer to the DANZ Monitoring Fabric 8.5 Hardware Compatibility List. The breakout cables listed in the Hardware Compatibility List are broken out automatically; manually entering the breakout command is not required.
GUI Procedure
To use the GUI to manually enable the use of multiple interfaces on a single switch port with a breakout cable, select Fabric > Interfaces, select Edit from the menu control for an interface, and use the settings on the Traffic page (Page 2) of the Edit Interface dialog.
To enable the use of multiple interfaces on a single switch port with a breakout cable, complete the following steps.
CLI Procedure
Use the breakout mode command to configure the breakout property for the current interface to configure the force breakout on an interface. When the config is applied, the interface, if it supports the breakout for the mode, is broken out into sub-interfaces based on the specified mode. Auto is the default option, which lets the switch automatically select the mode for the breakout. The following are the modes supported.
2x100G Breakout to 2 sub-interfaces of 100G each
2x200G Breakout to 2 sub-interfaces of 200G each
2x40G Breakout to 2 sub-interfaces of 40G each
2x50G Breakout to 2 sub-interfaces of 50G each
4x100G Breakout to 4 sub-interfaces of 100G each
4x10G Breakout to 4 sub-interfaces of 10G each
4x1G Breakout to 4 sub-interfaces of 1G each
4x25G Breakout to 4 sub-interfaces of 25G each
4x50G Breakout to 4 sub-interfaces of 50G each
8x10G Breakout to 8 sub-interfaces of 10G each
8x25G Breakout to 8 sub-interfaces of 25G each
8x50G Breakout to 8 sub-interfaces of 50G each
To verify the breakout-capable ports on the switch, enter the show switch interfaces command, which has the following syntax:
show switchswitch-nameinterfaces
To locate breakout-capable ports from the Controller, enter the following command:
Each breakout-capable port is identified by the string bsn-breakout-capable in the Supported Features column. In this example, ports ethernet49 through ethernet54 are breakout-capable.
Enter the enable and configure command to enter config mode, as in the following example.
controller-1> en
controller-1#
conf controller-1(config)#
Enter the config-switch submode and then the config-switch-if submode to enter the breakout command, as in the following example.
The breakout ports are named ethernetx/1 through ethernetx/4. The example output shows four 10G ports where there was previously a single 40G port (ethernet54/1 through ethernet54/4).
Verifying Switch Configuration
GUI Procedure
Use the Fabric > Interfaces option to view the interfaces table, which provides information about the configuration and activity on each interface of the switches connected to the DANZ Monitoring Fabric (DMF) controller.
CLI Procedure
To view the configuration or activity for a specific interface, use the show
switchswitchname interfaces command. The detail option provides additional information about the interface, including the up and down counts, indicating if the interface has been flapping. The output also indicates if the interface supports breakout interfaces.
To view the interface descriptions, use the show switch
switchname interface description command as shown in the following example.
controller-1(config-switch-if)# show switch DMF-DELIVERY-SWITCH-1 interface description
# Switch Name IF NameDescription
-|---------------------|------------|---------------|
1 DMF-DELIVERY-SWITCH-1 ethernet1100g-to-SFO
2 DMF-DELIVERY-SWITCH-1 ethernet2100g-to-NYC
Disabling the TX Direction on an Interface
DANZ Monitoring Fabric (DMF) supports disabling the TX direction of an interface, which is useful for example on a DMF switch whose filter interfaces are directly connected to a 40 GbE bidirectional tap using 40 GbE BiDi optics.
Note: when using bidirectional TAPs, either you need to use RX-only BiDi optics or you need to disable the TX direction on the DMF interfaces; otherwise, the optic transmits light back to the TAP and interferes with the production link. This is the case with all bidirectional TAPs: the outputs of the bidirectional TAPs should be connected to transceivers that don’t transmit because there is no way for the TAP to filter out light coming back from the packet broker.
If a filter interface is connected to a bidirectional TAP using a BiDi optic, you should configure that filter interface with the disable-xmit command to avoid optical interference with the TAP: