CloudVision DMF Integration

This chapter describes integrating CloudVision with the DANZ Monitoring Fabric (DMF).

Overview

In a typical CloudVision-DMF integration deployment, CloudVision Portal (CVP) deploys alongside the DANZ Monitoring Fabric (DMF). The DMF Controller communicates with CVP to retrieve its managed device inventory and configures port mirroring sessions on any CVP-managed production devices that are Arista Extensible Operating System (EOS) switches.

Configuration on the DMF Controller provides the information necessary to communicate with CVP: the CVP hostname or IP address and user credentials.

Policy configuration on the DMF Controller specifies what to monitor in the production network managed by CVP, such as the production switches, the switch interfaces to monitor traffic from, and the direction of mirrored traffic (bidirectional, ingress, or egress). In addition, the configuration on the DMF Controller can define whether to use a Switch Port Analyzer (SPAN) session or a Layer-2 Generic Routing Encapsulation (L2GRE) tunnel session on a CVP-managed device. When using SPAN, the DMF configuration includes the switch interface to monitor traffic. When using L2GRE, the DMF configuration includes monitoring traffic to the Tunnel End Point (TEP).

Figure 1. Simple CloudVision-DMF Integration Deployment

The preceding figure illustrates a simple CloudVision-DMF integration configuration where CloudVision Portal and DMF can communicate with each other. DMF monitors traffic from CVP directly to one of its fabric switches (a filter switch), as indicated by the red arrow labeled “SPAN.” DMF also monitors traffic from CVP to a TEP configured on the fabric using an L2GRE tunnel, as indicated by the green arrow labeled “L2GRE Tunnel". Since DMF initiates monitoring using policy configuration, the policies monitoring CVP will handle the traffic according to their configuration, for example, forwarding it to a delivery interface. This feature enables the automation of the creation, modification, and deletion of filter interfaces and tunnel interfaces in DMF and mirroring sessions on CVP-managed devices.

Compatibility Requirements

EOS Platform Compatibility

Any platform supporting port mirroring or mirroring to a GRE tunnel specifying the GRE key is supported; refer to EOS Port Mirroring TOI and EOS Mirroring to GRE Tunnel TOI for more details.

CloudVision Compatibility

On-premise 2024.2.0 and newer is recommended.

CloudVision Requirements

The user configured in DMF for CVP integration must have sufficient permissions in CVP. The minimum permissions required are:
- Devices: Read access to inventory management.
- gNMI: Read and write access to the gNMI service.
Register the devices that DMF will monitor for use in Studios using the Inventory and Topology Studio.

CloudVision DMF Integration using the CLI

To integrate with CloudVision Portal, configure a CVP instance in the DMF Controller, enabling communication between CVP and DMF. The CVP hostname or IP address must be reachable from the DMF Controller. If CVP is a multi-node system, using a hostname that will resolve to the primary node is recommended to maintain the connection in case of a primary node failure. The user in the CVP integration configuration must have at least the permissions in CloudVision as outlined in CloudVision Requirements.

Configure using the CLI

Configure a CVP instance on the DMF Controller using the following series of commands:

dmf-controller(config)# cvp cvp_instance_name
dmf-controller(config-cvp)# host-name cvp_hostname_or_ip
dmf-controller(config-cvp)# username username
dmf-controller(config-cvp)# password password

Add a description to the CVP instance using the description command, as required.

dmf-controller(config-cvp)# description description_of_cvp_instance

Refresh the connection between DMF and CVP with the sync command, which sends a request to CVP to re-authenticate the connection and to re-fetch the inventory:

dmf-controller(config)# sync cvp cvp_instance_name

To use L2GRE tunnels in the integration, enable tunneling in the DMF Controller and set the match mode to one of the following that is compatible with tunneling: full-match or l3-l4-offset-match. Configure tunnel endpoints to allow monitoring from CVP to DMF using an L2GRE tunnel; add a tunnel endpoint to a policy configuration or to a CVP integration instance's configuration to optionally define a default tunnel endpoint for this instance.

Before the DMF 8.5 release, the following command string was mandatory:

dmf-controller(config)# tunnel-endpoint tep_name switch fabric_switch fabric_switch_interface ip-address tep_ip mask subnet_mask gateway gateway_ip

However, starting with DMF 8.5.0, the mask and gateway parameters in the tunnel-endpoint command are now optional. Thus, configure a tunnel endpoint using the following command:

dmf-controller(config)# tunnel-endpoint tep_name switch fabric_switch fabric_switch_interface ip-address tep_ip

To set a default tunnel endpoint for a CVP integration instance, use the following commands:

dmf-controller(config)# cvp cvp_instance_name
dmf-controller(config-cvp)# default-tunnel-endpoint tep_name

To remove a default tunnel endpoint for a CVP integration instance, use the following commands:

dmf-controller(config)# cvp cvp_instance_name
dmf-controller(config-cvp)# no default-tunnel-endpoint tep_name

Starting with the DMF 8.6.0 release, a configuration flag called preserve-mirror-sessions per CVP instance indicates whether mirroring sessions will be preserved for the CVP instance when uninstalling DMF policies configured with it. By default, the flag is false, meaning existing mirroring sessions are automatically removed if the relevant DMF policies are uninstalled.

Enable preserving mirroring sessions using the preserve-mirror-sessions command.

dmf-controller(config-cvp)# preserve-mirror-sessions

Conversely, disable preserving mirroring sessions (default behavior) using the no preserve-mirror-sessions command.

dmf-controller(config-cvp)# no preserve-mirror-sessions

Monitoring Configuration in Policies

DMF uses policies to create, update, or remove the monitoring of CVP-managed devices. DMF supports monitoring multiple CVP instances, switches, and interfaces as mirroring sources in a single policy or across policies. Configure the mirrored traffic direction to one of the following settings:

bidirectional (default)
ingress
egress

After enabling CVP integration in a DMF policy (i.e., adding a CVP instance as a traffic source), the DMF Controller will automatically create filter interfaces and tunnel interfaces, with origination "auto-generated." A mirroring session is automatically created on the CVP-managed switch; DMF does this via the mirroring Studio and the change control process on CVP.

Note: If a DMF-managed mirroring session exists on a switch for one DMF policy with identical sources and the same destination as needed for another DMF policy, both policies use the same mirroring session.

Add a CVP instance as a traffic source in a DMF policy using the following series of commands:

dmf-controller(config)# policy policy_name
dmf-controller(config-policy)# filter-cvp cvp_instance_name 
dmf-controller(config-policy-cvp)#

To monitor traffic using SPAN, configure a SPAN interface (on the CVP-managed device) as the destination in a DMF policy, along with the source interfaces (on the CVP-managed device) and optionally the direction for each source interface.

Select the switch interfaces on CVP-managed devices individually as source interfaces to a SPAN interface on that switch using the following series of commands where including the direction is optional :

dmf-controller(config-policy-cvp)# device device_hostname
dmf-controller(config-policy-cvp-device)# src-interface source_interface 
span-interface span_interface direction ingress | egress | bidirectional

Select the switch interfaces on CVP-managed devices as source interfaces using an interface range to a SPAN interface on that switch using the following series of commands where including the direction is optional:

dmf-controller(config-policy-cvp)# device device_hostname
dmf-controller(config-policy-cvp-device)# src-interface-range start start_of_range 
end end_of_range span-interface span_interface direction ingress | egress | bidirectional

To monitor traffic using L2GRE tunneling, choose from two options: (1) configure a tunnel endpoint (in DMF) as the destination in a DMF policy along with the source interfaces (on the CVP-managed device) and optionally the direction for each source interface, or (2) omit the destination in a DMF policy along with configuring the source interfaces (on the CVP-managed device) and optionally the direction for each source interface.

A GRE tunnel source IP can be optionally configured on DMF as the tunnel source IP on the CVP-managed device to overcome reachability issues due to possible Reverse Path Forwarding (RPF) checks between the CVP and DMF deployment. By default, the tunnel source IP is the switch’s management IP.

Select the switch interfaces on CVP-managed devices individually as source interfaces to a tunnel endpoint configured in DMF using the following series of commands where the direction is optional:

dmf-controller(config-policy-cvp)# device device_hostname
dmf-controller(config-policy-cvp-device)# src-interface source_interface gre-tunnel-src src_ip
gre-tunnel-endpoint tep_name direction ingress | egress | bidirectional

The src-interface-range command is also supported for GRE tunnel configuration in a policy.

The following example illustrates two DMF policies’ configuration, where testPolicy1 is monitoring traffic from Ethernet1 on the CVP-managed device (production switch) called dev1 in the CVP instance, test, to Ethernet2 on the same device, using SPAN, and forwarding the traffic to the delivery interface called tool1; testPolicy2 is monitoring traffic from Ethernet5 on the CVP-managed device called dev2 in the same CVP instance, test, to the default tunnel endpoint called TEP1 defined in the CVP integration instance configuration, using L2GRE tunneling, and forwarding the traffic to the delivery interface called tool2.

! cvp
cvp test
default-tunnel-endpoint TEP1
hashed-password abc123
host-name test.arista.com
user-name cvpadmin

! policy
policy testPolicy1
 action forward
 delivery-interface tool1
 1 match any
 filter-cvp test
 !
 device dev1
 src-interface Ethernet1 span-interface Ethernet2

policy testPolicy2
action forward
delivery-interface tool2
1 match any
filter-cvp test
!
device dev2
src-interface Ethernet5

Suppose you remove the configuration to monitor CVP-managed devices from the DMF Controller. In that case, the system removes the corresponding auto-generated filter interfaces and tunnel interfaces from DMF and deletes the auto-created mirroring sessions on the switch.

To stop monitoring a source interface or a range of source interfaces on a CVP-managed device, remove its configuration from a DMF policy using the following series of commands:

dmf-controller(config-policy-cvp-device)# no src-interface source_interface
dmf-controller(config-policy-cvp-device)# no src-interface-range start start_of_range end end_of_range

Stop monitoring a device in a CVP instance in a DMF policy.

dmf-controller(config-policy-cvp)# no device device_hostname

Stop monitoring a CVP instance in a DMF policy and all its devices.

dmf-controller(config)# policy policy_name
dmf-controller(config-policy)# no filter-cvp cvp_instance_name

To disable integration with a CVP instance, remove its CVP integration instance configuration and remove it from all DMF policies using the following series of commands:

dmf-controller(config)# no cvp cvp_instance_name
dmf-controller(config)# policy policy_name
dmf-controller(config-policy)# no filter-cvp cvp_instance_name

Show Commands

After configuring a CVP instance, the show cvp cvp_instance_name command displays the configuration and connection status information.

dmf-controller(config)# show cvp test
# CVPHostnameState Last Update Time Detail State Version
-|----|---------------|---------|------------------------------|------------|--------|
1 test test.arista.com connected 2023-12-13 05:17:28.512000 UTC connected2024.1.0

The show cvp cvp_instance_name detail command displays detailed status information about the integration.

dmf-controller(config)# show cvp test detail
CVP: test
Hostname : test.arista.com
State: connected
Last Update Time : 2023-12-13 05:17:54.072000 UTC
Detail State : connected
Version: 2024.1.0

The show cvp cvp_instance_name alert and show cvp cvp_instance_name error commands display runtime warnings and alerts, and errors, if any.

dmf-controller(config)# show cvp cvp_instance_name alert
dmf-controller(config)# show cvp cvp_instance_name error

Note: It is possible to specify all in the above show cvp commands to see the information for all CVP integration instances on the DMF Controller; for example, show cvp all alert.

The show cvp cvp_instance_name device device_hostname command displays the device inventory in the CVP deployment; only EOS devices are supported. Using all is possible for the CVP instance name and the device hostname in the show cvp cvp_instance_name device device_hostname command.

dmf-controller(config)# show cvp test device all
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Device Inventory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# CVPDevice FQDNStreaming Model Software IP Address MAC AddressDevice ID
-|----|------|-----------------|---------|-------|--------|------------|--------------------------|-----------|
1 test dev123 dev123.arista.com activeABC-123 4.31.2F10.10.10.10aa:bb:cc:dd:ee:ff (Arista) DEV123

The show cvp cvp_instance_name device device_hostname interface command includes a list of all the device interfaces.

dmf-controller(config)# show cvp test device all
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Device Inventory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# CVPDevice FQDNStreaming Model Software IP Address MAC AddressDevice ID
-|----|------|-----------------|---------|-------|--------|------------|--------------------------|-----------|
1 test dev123 dev123.arista.com activeABC-123 4.31.2F10.10.10.10aa:bb:cc:dd:ee:ff (Arista) DEV123

~~~~ Device Interfaces ~~~~
#CVPDevice Interface
--|----|------|-----------|
1test dev123 Ethernet1
2test dev123 Ethernet2
3test dev123 Ethernet3

After configuring a DMF policy to take a CVP instance as a traffic source, the show fabric errors command displays any errors with the integration relating to monitoring, if any.

dmf-controller(config)# show fabric errors

In addition, the DMF Controller can show the current mirroring sessions configured on a CVP-managed device used to confirm the current state of a mirroring session created by DMF (thus, managed by DMF) or otherwise (non-DMF-managed sessions are only displayed in the detail command). There are three commands to display the mirroring state on a CVP-managed device in varying levels of detail, as follows:

1) The show cvp cvp_instance_name device device_hostname session command displays only the mirroring sessions managed by DMF.

dmf-controller(config)# show cvp test device dev123 session
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ L2-GRE Mirroring Sessions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# CVPHostname Tunnel Endpoint Programmed in hardware Tunnel Src Tunnel Dst Src Interface Src Link Status Src Direction
-|----|--------|---------------|----------------------|----------|----------|-------------|---------------|-------------|
1 test dev123 unknown3.3.3.34.4.4.4Ethernet2 unspecified bidirectional

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ SPAN Mirroring Sessions~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# CVPHostname SPAN Interface SPAN Status Programmed Src Interface Src Link Status Src Direction
-|----|--------|--------------|-----------|----------|-------------|---------------|-------------|
1 test dev123 Ethernet5uptrue Ethernet4 upbidirectional

2) The show cvp cvp_instance_name device device_hostname session brief command displays a summary of the state of the mirroring sessions managed by DMF.

dmf-controller(config)# show cvp cvp_instance_name device device_hostname session brief

3) The show cvp cvp_instance_name device device_hostname session detail command displays all the mirroring sessions on the device, both managed by DMF and otherwise, as well as the name of each session.

dmf-controller(config)# show cvp cvp_instance_name device device_hostname session detail

CloudVision DMF Integration using the GUI

Using the GUI

Navigate to Integration > CloudVision Portal.

Select Add CloudVision Portal to create a new CVP integration instance.

Figure 2. DMF Integration CloudVision Portal

Enter the CVP integration instance configuration details and click Submit.

If there are any warnings or errors with the CVP integration instance, click the alarm bell icon to view more details.

Select the CVP instance name to view its details, including the device inventory, port mirroring status, and port mirroring entries in DMF policies.

Note: The Port Mirroring Entries table will not be visible if no policies using this CVP instance as a traffic source have been configured.

To start monitoring traffic from the production network:

Navigate to Monitoring > Policies.
Select Create Policy and add CVP instances as traffic sources.
Select Add Row to configure monitoring, such as the device interfaces to monitor, the monitor type (e.g., SPAN or L2GRE), the mirrored traffic direction, and the destination.

Policy configuration details related to CVP integration appear in the policy’s Configuration Details page.

To edit the CVP monitoring configuration in a policy, click Edit and select 1 Entry on the Edit Policy page, make the required changes, and click Save Policy.

Limitations

The following limitations apply to the DANZ Monitoring Fabric (DMF) Controller and CloudVision integration.

Modifying or deleting auto-generated filter interfaces for CVP integration or adding them manually to non-CloudVision policies will result in unexpected behavior.
If two or more DMF policies have overlapping CVP monitoring configurations with the same destination, the configuration should be the same in these policies if unexpected traffic is undesired. For example, if a policy has source interfaces A and B on a device in a CVP instance with a SPAN interface as the destination, and if another policy has one source interface, A, on the same device in the same CVP instance with the same SPAN interface as the former policy, then the latter policy will unexpectedly receive traffic from B as well as the expected source A. This is because the mirroring session in the production switch is reused for both policies.
An error state in the CVP deployment may prevent DMF from configuring mirroring sessions on the production switches. If DMF encounters such a failure, a fabric error will be displayed, and user action in CVP is required. Examples of such a state include the production switch not being in compliance in CVP, in which case it needs to be brought into compliance, or if there are any pending change controls in CVP, they must be addressed. After taking the appropriate corrective action in CVP, deactivate and reactivate the DMF policy with the error (or delete and recreate it) to retry configuring mirroring.

Troubleshooting

If there are any fabric errors in creating a mirroring session on a CVP-managed device that specify that user action in CVP is required, take the appropriate corrective action in CVP. Next, delete the CVP monitoring config from the policy and reconfigure it, or delete the policy and then recreate it.
If there is a message in the show cvp alert command output stating a session failed to be updated and an authentication error message in /var/log/vsphere-extension/vsphere-extension.log containing GNMI7001…Status unauthenticated, run the sync cvp command, and then deactivate and reactivate the relevant DMF policy.
If there is a fabric error containing No change controls to approve and execute, a possible cause is that the device had not been registered for use in Studios in CVP using the Inventory and Topology studio. If so, register the device in CVP and delete and recreate the DMF policy.

Resources

Please refer to tunneling and policies in this guide for further information.

DANZ Monitoring Fabric User Guide