Integrating vCenter with the DANZ Monitoring Fabric
This chapter describes integrating VMware vCenter with the DANZ Monitoring Fabric (DMF) and monitoring Virtual Machines (VM) in the vCenter.
Overview
The DANZ Monitoring Fabric (DMF) allows the integration and monitoring of VMs in a VMware vCenter cluster. After integrating a vCenter with the DMF fabric, use DMF policies to select different types of traffic from specific VMs and apply managed services, such as deduplication or header slicing, to the selected traffic.
Currently, DMF supports the following versions of VMware vCenter for monitoring:
- vCenter Server 6.5.0
- vCenter Server 6.7.0
- vCenter Server 7.0.0
- vCenter Server 8.0.0
The DANZ Monitoring Fabric provides two options to monitor a VMware vCenter cluster:
- Monitoring using span ports: This method monitors VMware vCenter clustering using a separate monitoring network. The advantage of this configuration is that it has no impact on the production network and has a minimal effect on compute node CPU performance. However, in this configuration, each compute node must have a spare NIC to monitor traffic.
The following figure illustrates the topology used for local SPAN configuration:
- Monitoring using ERPAN/L2GRE tunnels: Use Remote SPAN (ERSPAN) to monitor VMs running on the ESX hosts within a vCenter instance integrated with DMF. ERSPAN monitors traffic to and from VMs anywhere in the network and does not require a dedicated physical interface card on the ESX host. However, ERSPAN can affect network performance, especially when monitoring VMs connected to the DMF Controller over WAN links or production networks with high utilization.
Using SPAN to Monitor VMs
This section describes the configuration required to integrate the DANZ Monitoring Fabric (DMF) Controller with one or more vCenter instances and to monitor traffic from VMs connected to the VMware vCenter after integration.
The following figure illustrates the topology required to integrate a vCenter instance with the monitoring fabric and deliver the traffic selected by DMF policies to specified delivery ports connected to different monitoring tools.
When integrated with vCenter, the DMF Controller uses Link Layer Discovery Protocol (LLDP) to automatically identify the available filter interfaces connected to the vCenter instance.
Using ERSPAN to Monitor VMs
The procedure for deploying ERSPAN is similar to SPAN but requires an additional step to define the tunnel endpoints used on the DMF network to terminate the ERSPAN session.
Configuration Summary for vCenter Integration
The following procedure summarizes the high-level steps required to integrate the vCenter and monitor traffic to or from selected VMs:
Defining a Tunnel Endpoint
Predefine the tunnel endpoints for creating tunnels when monitoring VMware vCenter traffic using either the GUI or the CLI.
GUI Procedure
To manage tunnel endpoints in the GUI, select
.This page lists the tunnel endpoints that are already configured and provides information about each endpoint.
- Name: Type a descriptive name for the endpoint.
- Switch: Select the DMF switch from the selection list for the configured endpoint interface.
- Interface: Select the interface from the selection list for the endpoint.
- Gateway: Type the address of the default gateway.
- IP Address: Type the IP address of the endpoint.
- Mask: Type the subnet mask for the endpoint.
CLI Procedure
controller-1(config)# tunnel-endpoint <name> switch <switch> <interface> ip-address <address> mask
<mask> gateway <address>
controller-1(config)# tunnel-endpoint ERSPAN switch CORE-SWITCH ethernet7 ip-address 172.27.1.1
mask 255.255.255.0 gateway 172.27.1.2
The IP address assigned to this endpoint is 172.27.1.1, and the next hop address for connecting to the vCenter via ERSPAN is 172.27.1.2.
Using the GUI to Integrate a vCenter Instance
This page displays information about the vCenter instances integrated with DMF. To add a vCenter instance for integration with DMF, complete the following steps:
Using a vCenter Instance as the Traffic Source in a DMF Policy
Proceed to the
page.Using the CLI to Integrate a vCenter Instance
To configure the DMF Controller for monitoring VMs on a locally connected vCenter instance, complete the following steps:
Using the GUI to View vCenter Configuration
DANZ Monitoring Fabric (DMF) displays the vCenter Info page.
- Hosts
- Virtual Switches
- Physical Connections
- Virtual Machines
- Network Host Connection Details
Using the CLI to View vCenter Configuration
controller-1# show vcenter
#vCenter Name vCenter Host Name or IP Last vCenter Update Time Detail State vSphere Version
--|------------|-----------------------|------------------------------|----------------------------|---------------|
1vc-10-9-0-75 10.9.0.75 2017-09-0918:02:35.980000 PDTConnected and authenticated. 6.5.0
2vc-10-9-0-76 10.9.0.76 2017-09-0918:02:36.488000 PDTConnected and authenticated. 6.5.0
3vc-10-9-0-77 10.9.0.77 2017-09-0918:02:35.908000 PDTConnected and authenticated. 6.0.0
4vc-10-9-0-78 10.9.0.78 2017-09-0918:02:33.507000 PDTConnected and authenticated. 6.5.0
5vc-10-9-0-79 10.9.0.79 2017-09-0918:02:32.248000 PDTConnected and authenticated. 6.5.0
6vc-10-9-0-80 10.9.0.80 2017-09-0918:02:32.625000 PDTConnected and authenticated. 6.0.0
7vc-10-9-0-81 10.9.0.81 2017-09-0918:02:34.672000 PDTConnected and authenticated. 6.0.0
8vc-10-9-0-82 10.9.0.82 2017-09-0918:02:33.008000 PDTConnected and authenticated. 6.0.0
9vc-10-9-0-83 10.9.0.83 2017-09-0918:02:30.011000 PDTConnected and authenticated. 6.0.0
10 vc-10-9-0-84 10.9.0.84 2017-09-0918:02:33.024000 PDTConnected and authenticated. 6.5.0
11 vc-10-9-0-85 10.9.0.85 2017-09-0918:02:34.827000 PDTConnected and authenticated. 6.0.0
12 vc-10-9-0-86 10.9.0.86 2017-09-0918:02:35.164000 PDTConnected and authenticated. 6.0.0
13 vc-10-9-0-87 10.9.0.87 2017-09-0918:02:38.042000 PDTConnected and authenticated. 6.5.0
14 vc-10-9-0-88 10.9.0.88 2017-09-0918:02:37.212000 PDTConnected and authenticated. 6.0.0
15 vc-10-9-0-89 10.9.0.89 2017-09-0918:02:33.436000 PDTConnected and authenticated. 6.5.0
controller-1#
controller-1# show vcenter vc-10-9-0-75
#vCenter Name vCenter Host Name or IP Last vCenter Update Time Detail State vSphere Version
--|------------|-----------------------|------------------------------|----------------------------|---------------|
1vc-10-9-0-75 10.9.0.75 2017-09-0918:02:44.698000 PDTConnected and authenticated. 6.5.0
controller-1#
controller-1# show vcenter vc-10-9-0-75 detail
vCenter Name : vc-10-9-0-75
vCenter Host Name or IP : 10.9.0.75
Last vCenter Update Time : 2017-09-09 18:02:49.463000 PDT
Detail State : Connected and authenticated.
vSphere Version : 6.5.0
controller-1#
controller-1# show vcenter vc-10-9-0-75 error
vCenter Name : vc-10-9-0-75
vCenter Host Name or IP : 10.9.0.75
State : connected
Detail State : Connected and authenticated.
Detailed Error Info :
controller-1#
Wildcard Tunnels for VMware vCenter Monitoring
The current implementation of VMware vCenter creates one tunnel interface from every ESXi host to DMF.
Using a wildcard tunnel on DMF for VMware vCenter reduces the number of tunnels created.
Platform Compatibility
This feature is only compatible with switches that support wildcard tunneling.
Configuration
Configure wildcard tunnels using the CLI or the GUI.
Using the CLI to Create Wildcard Tunnels
The CLI construct wildcard-tunnels
is available as a configuration option when configuring a VMware vCenter in DANZ Monitoring Fabric (DMF), as shown below:
cluster |
Configure tunnel-endpoint for cluster |
default-tunnel-endpoint |
Configure tunnel endpoints |
description |
Describe this vCenter |
hashed-password |
Set the vCenter password (to log into vCenter) |
host-name |
Set the vCenter hostname |
mirror-type |
Set the vCenter vm monitoring mode |
mirrored-packet-length |
Set the mirrored packet length |
password |
Set the vCenter password (to log into vCenter) |
sampling-rate |
Set the packet sampling rate |
user-name |
Set the vCenter user name (to log into vCenter) |
vm-monitoring |
Enter vm-monitoring config submode |
wildcard-tunnels |
Enable wildcard tunnels |
Enable wildcard tunnels by setting the above leaf parameter, as shown in the following example of vCenter configuration on the Controller node.
dmf-controller-1(config)# vcenter VC1
dmf-controller-1(config-vcenter)# wildcard-tunnels
dmf-controller-1(config-vcenter)# show this
! vcenter
vcenter VC1
wildcard-tunnels
dmf-controller-1(config-vcenter)#
Similarly, disable wildcard tunnels by issuing the no
command as shown below:
dmf-controller-1(config-vcenter)# show this
! vcenter
vcenter VC1
wildcard-tunnels
dmf-controller-1(config-vcenter)# no wildcard-tunnels
dmf-controller-1(config-vcenter)# show this
! vcenter
vcenter VC1
dmf-controller-1(config-vcenter)#
Show Commands
There is no specific show command for wildcard tunnels; however, check them in the vCenter running config. In addition, the show tunnels
command shows the tunnels created for the selected vCenter configuration with a wildcard remote IP address.
Troubleshooting
Verify errors and warnings are clear using the show fabric errors
and show fabric warnings
commands. The show
tunnels
command displays tunnels created based on the vCenter configuration on the Controller with a wildcard remote IP address. Use the show switch <name> table gre-tunnel
command to display tunnels programmed on the switch.
Using the GUI to Create Wildcard Tunnels
Use the DANZ Monitoring Fabric (DMF) GUI to create wildcard tunnels as outlined below.
Click the Menu icon.
Limitations
Select Broadcom® switch ASICs support wildcard tunnels; ensure your switch model supports this feature before configuring it for vCenter.
Please refer to the Platform Compatibility section for more information.
Minimum Permissions for Non-admin Users
For a non-admin user to add, remove, edit, or monitor a vCenter via the DANZ Monitoring Fabric (DMF), the privilege level assigned to the non-admin user should be VSPAN operation. To assign VSPAN operation privileges to a user, perform the following steps:
Monitor VMware vCenter Traffic by VM Names
Match VMware vCenter-specific information in the policy. Specifically, this feature matches traffic using VMware vCenter Virtual Machine (VM) names and requires DANZ Monitoring Fabric (DMF) vCenter integration.
Using the CLI to Monitor vCenter Traffic by VM Names
Configuration
dmf-controller-1(config)# policy v1
dmf-controller-1(config-policy)# action forward
dmf-controller-1(config-policy)# filter-interface filter-interface
dmf-controller-1(config-policy)# delivery-interface delivery-interface
dmf-controller-1(config-policy)# filter-vcenter vcenter-name
dmf-controller-1(config-policy)# 1 match ip src-vm-name vm-name dst-vm-name vm-name
dmf-controller-1(config-policy)# 2 match ip6 src-vm-name vm-name
Show Commands
show running-config policy policy
name
command to display the configuration.
dmf-controller-1# show running-config policy v1
! policy
policy v1
action forward
delivery-interface delivery-interface
filter-interface filter-interface
filter-vcenter vcenter-name
1 match ip src-vm-name vm-name dst-vm-name vm-name
2 match ip6 src-vm-name vm-name
show policy policy name
command displays the policy information, including stats.
dmf-controller-1# show policy v2
Policy Name: v2
Config Status: active - forward
Runtime Status : installed
Detailed Status: installed - installed to forward
Priority : 100
Overlap Priority : 0
# of switches with filter interfaces : 1
# of switches with delivery interfaces : 1
# of switches with service interfaces: 0
# of filter interfaces : 1
# of delivery interfaces : 1
# of core interfaces : 0
# of services: 0
# of pre service interfaces: 0
# of post service interfaces : 0
Push VLAN: 5
Post Match Filter Traffic: -
Total Delivery Rate: -
Total Pre Service Rate : -
Total Post Service Rate: -
Overlapping Policies : none
Component Policies : none
Installed Time : 2023-12-21 19:00:39 UTC
Installed Duration : 50 minutes, 11 secs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Match Rules ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Rule
-|--------------------------------------------------------------------------|
1 1 match ip src-vm-name DMF-RADIUS-SERVER-1 dst-vm-name DMF-TACACS-SERVER-1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Filter Interface(s)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# DMF IF Switch IF NameState Dir Packets Bytes Pkt Rate Bit Rate Counter Reset Time
-|----------------|----------|----------|-----|---|-------|-----|--------|--------|------------------------------|
1 span_from_arista DELL-S4048 ethernet20 uprx0 0 0-2023-12-21 19:00:39.941000 UTC
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Delivery Interface(s)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# DMF IF Switch IF NameState Dir Packets Bytes Pkt Rate Bit Rate Counter Reset Time
-|------------|----------|------------|-----|---|-------|-----|--------|--------|------------------------------|
1 ubuntu-tools DELL-S4048 ethernet50/2 uptx0 0 0-2023-12-21 19:00:39.941000 UTC
~ Service Interface(s) ~
None.
~ Core Interface(s) ~
None.
~ Failed Path(s) ~
None.
show vcenter vcenter name endpoint
command displays the vCenter VM information, including networks.
dmf-controller-1# show vcenter vcenter1 endpoint
#vCenter Name VM Name ESXi Host Name Network Interface Name MAC AddressIP Address Virtual Switch Portgroup Power State
--|------------|---------|--------------|----------------------|--------------------------|------------------------------------------|--------------|-------------|-----------|
1vcenter1 ub-11-216 10.240.155.216 Network adapter 100:50:56:8b:4d:03 (VMware) 1.1.11.216/24, fe80::250:56ff:fe8b:4d03/64 DVS-DMFvlan11powered-on
2vcenter1 ub-12-216 10.240.155.216 Network adapter 100:50:56:8b:72:a0 (VMware) 1.1.12.216/24, fe80::250:56ff:fe8b:72a0/64 DVS-DMFvlan12powered-on
3vcenter1 ub-13-216 10.240.155.216 Network adapter 100:50:56:8b:c0:06 (VMware) 1.1.13.216/24, fe80::250:56ff:fe8b:c006/64 DVS-DMFvlan-10 powered-on
4vcenter1 ub-14-216 10.240.155.216 Network adapter 100:50:56:8b:d1:d9 (VMware) 1.1.14.216/24, fe80::250:56ff:fe8b:d1d9/64 DVS-DMFvlan-10 powered-on
Using the GUI to Monitor vCenter Traffic by VM Names
Troubleshooting
Fabric errors and warnings are very useful for troubleshooting this feature.
dmf-controller-1# show fabric warnings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Policy related warning~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Policy Name Warning
-|-----------|------------------------------------------------------------------------------------------------------------|
1 v1No IP found for VMs [ub-15-216, ub-216-multinic, ub-217-vlan10, ub-14-216, ub-11-216] associated with policy
dmf-controller-1# show fabric warnings
~~~~~~~~~~~~~~~~~~~ Policy related warning ~~~~~~~~~~~~~~~~~~~
# Policy Name Warning
-|-----------|-----------------------------------------------|
1 v1No vCenter associated to policy with VM matches
Limitations
- This feature only works with vCenter integration and a direct SPAN from customer switch with ESXi traffic.
- VM interface IP addresses connected to dvs will be added to policy matches.
- May use extra TCAM entries in case the management network uses dvs.
- Vmkernal names cannot be matched in the policy.
- Extra TCAM entries may be used for the removed/modified IP addresses when the VM IP addresses are updated in the vCenter.
- TCAM entry will be added for all the IP addresses when a VM Name with multiple vNIC (multiple IP addresses) matches in the policy.
- VM Names cannot be matched with the MAC option in the policy.
- If the vCenter becomes disconnected, policies associated with the VM names may not get correct matches or traffic.