In assisting customers with a wide variety of NSX related issues, I’m constantly pulling up cheat sheets for frequently used commands. This page organizes my favorite NSX ESXi network commands into troubleshooting reference.
[root@esx01:~] esxcli network nic list Name PCI Device Driver Admin Status Link Status Speed Duplex MAC Address MTU Description ------ ------------ ------ ------------ ----------- ----- ------ ----------------- ---- -------------------------------------------------------------- vmnic0 0000:02:00.0 e1000 Up Up 1000 Full 00:50:56:01:44:05 1600 Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) vmnic1 0000:02:01.0 e1000 Up Up 1000 Full 00:50:56:01:10:b9 1600 Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) vmnic2 0000:02:02.0 e1000 Up Up 1000 Full 00:50:56:01:10:bb 1600 Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) vmnic3 0000:02:03.0 e1000 Up Up 1000 Full 00:50:56:01:10:bc 1500 Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) vmnic4 0000:02:04.0 e1000 Up Down 0 Half 00:50:56:01:10:c1 1500 Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) vmnic5 0000:02:05.0 e1000 Up Down 0 Half 00:50:56:01:10:c2 1500 Intel Corporation 82545EM Gigabit Ethernet Controller (Copper)
[root@esx01:~] esxcli network ip interface ipv4 get Name IPv4 Address IPv4 Netmask IPv4 Broadcast Address Type Gateway DHCP DNS ----- --------------- ------------- --------------- ------------ ------------- -------- vmk0 192.168.110.81 255.255.255.0 192.168.110.255 STATIC 192.168.110.2 false vmk1 10.10.20.81 255.255.255.0 10.10.20.255 STATIC 0.0.0.0 false vmk10 192.168.110.184 255.255.255.0 192.168.110.255 STATIC 0.0.0.0 false vmk50 169.254.1.1 255.255.0.0 169.254.255.255 STATIC 0.0.0.0 false
[root@esx01:~] esxcli network nic stats get -n vmnic0 NIC statistics for vmnic0 Packets received: 345141698 Packets sent: 25667 Bytes received: 302727485119 Bytes sent: 4363390 Receive packets dropped: 0 Transmit packets dropped: 0 Multicast packets received: 0 Broadcast packets received: 0 Multicast packets sent: 0 Broadcast packets sent: 0 Total receive errors: 0 Receive length errors: 0 Receive over errors: 0 Receive CRC errors: 0 Receive frame errors: 0 Receive FIFO errors: 0 Receive missed errors: 0 Total transmit errors: 0 Transmit aborted errors: 0 Transmit carrier errors: 0 Transmit FIFO errors: 0 Transmit heartbeat errors: 0 Transmit window errors: 0
[root@esx01:~] esxcli network vswitch dvs vmware list
DSwitch-CAI-Management
Name: DSwitch-CAI-Management
VDS ID: b1 75 16 50 72 84 37 a8-3f 1b 18 e4 55 d0 97 0d
Class: etherswitch
Num Ports: 1536
Used Ports: 7
Configured Ports: 512
MTU: 1600
CDP Status: both
Beacon Timeout: -1
Uplinks: vmnic1, vmnic0
VMware Branded: true
DVPort:
Client: vmnic0
DVPortgroup ID: dvportgroup-16
In Use: true
Port ID: 630
Client: vmnic1
DVPortgroup ID: dvportgroup-16
In Use: true
Port ID: 631
Client: vmk0
DVPortgroup ID: dvportgroup-17
In Use: true
Port ID: 623
Client: vmk1
DVPortgroup ID: dvportgroup-17
In Use: true
Port ID: 624
[root@esx01:~] esxcli network vswitch standard list vSwitch0 Name: vSwitch0 Class: etherswitch Num Ports: 1536 Used Ports: 1 Configured Ports: 128 MTU: 1500 CDP Status: listen Beacon Enabled: false Beacon Interval: 1 Beacon Threshold: 3 Beacon Required By: Uplinks: Portgroups: VM Network
[root@esx01:~] vsish -e get /net/pNics/vmnic0/stats
device {
-- General Statistics:
Rx Packets:345156392
Tx Packets:25668
Rx Bytes:302737398474
Tx Bytes:4363560
Rx Errors:0
Tx Errors:0
Rx Dropped:0
Tx Dropped:0
Rx Multicast:0
Rx Broadcast:0
Tx Multicast:0
Tx Broadcast:0
Collisions:0
Rx Length Errors:0
Rx Over Errors:0
Rx CRC Errors:0
Rx Frame Errors:0
Rx Fifo Errors:0
Rx Missed Errors:0
Tx Aborted Errors:0
Tx Carrier Errors:0
Tx Fifo Errors:0
Tx Heartbeat Errors:0
Tx Window Errors:0
Module Interface Rx packets:345156392
Module Interface Tx packets:25668
Module Interface Rx dropped:0
Module Interface Tx dropped:0
-- Driver Specific Statistics:
rx_packets : 345185309
tx_packets : 25668
rx_bytes : 304130296574
tx_bytes : 4363560
rx_broadcast : 0
tx_broadcast : 0
rx_multicast : 0
tx_multicast : 0
rx_errors : 0
tx_errors : 0
tx_dropped : 0
multicast : 0
collisions : 0
rx_length_errors : 0
rx_over_errors : 0
rx_crc_errors : 0
rx_frame_errors : 0
rx_no_buffer_count : 0
rx_missed_errors : 0
tx_aborted_errors : 0
tx_carrier_errors : 0
tx_fifo_errors : 0
tx_heartbeat_errors : 0
tx_window_errors : 0
tx_abort_late_coll : 0
tx_deferred_ok : 0
tx_single_coll_ok : 0
tx_multi_coll_ok : 0
tx_timeout_count : 0
tx_restart_queue : 0
rx_long_length_errors : 0
rx_short_length_errors : 0
rx_align_errors : 0
tx_tcp_seg_good : 0
tx_tcp_seg_failed : 0
rx_flow_control_xon : 0
rx_flow_control_xoff : 0
tx_flow_control_xon : 0
tx_flow_control_xoff : 0
rx_long_byte_count : 304130296574
rx_csum_offload_good : 341712820
rx_csum_offload_errors : 0
alloc_rx_buff_failed : 0
tx_smbus : 0
rx_smbus : 0
dropped_smbus : 0
[root@esx01:~] vsish -e get /net/pNics/vmnic0/properties
properties {
Driver Name:e1000
Driver Version:8.0.3.1-NAPI
Driver Firmware Version:N/A
System Device Name:vmnic0
Module Interface Used By The Driver:vmklinux
Device Hardware Cap Supported:: 0x602c032b -> VMNET_CAP_SG VMNET_CAP_IP4_CSUM VMNET_CAP_HIGH_DMA VMNET_CAP_TSO VMNET_CAP_HW_TX_VLAN VMNET_CAP_HW_RX_VLAN VMNET_CAP_SG_SPAN_PAGES VMNET_CAP_IP6_CSUM VMNET_CAP_TSO256k VMNET_CAP_TSO6_EXT_HDRS VMNET_CAP_SCHED
Device Hardware Cap Activated:: 0x602c032b -> VMNET_CAP_SG VMNET_CAP_IP4_CSUM VMNET_CAP_HIGH_DMA VMNET_CAP_TSO VMNET_CAP_HW_TX_VLAN VMNET_CAP_HW_RX_VLAN VMNET_CAP_SG_SPAN_PAGES VMNET_CAP_IP6_CSUM VMNET_CAP_TSO256k VMNET_CAP_TSO6_EXT_HDRS VMNET_CAP_SCHED
Device Software Cap Activated:: 0x10900000 -> VMNET_CAP_TSO6 VMNET_CAP_RDONLY_INETHDRS VMNET_CAP_IP6_CSUM_EXT_HDRS
Device Software Assistance Activated:: 0 -> No matching defined enum value found.
PCI Segment:0
PCI Bus:2
PCI Slot:0
PCI Fn:0
Device NUMA Node:4294967295
PCI Vendor:0x8086
PCI Device ID:0x100f
Link Up:1
Operational Status:1
Administrative Status:1
Full Duplex:1
Auto Negotiation:0
Speed (Mb/s):1000
Uplink Port ID:0x03000002
Flags:: 0x41e0e -> DEVICE_PRESENT DEVICE_OPENED DEVICE_EVENT_NOTIFIED DEVICE_SCHED_CONNECTED DEVICE_USE_RESPOOLS_CFG DEVICE_RESPOOLS_SCHED_ALLOWED DEVICE_RESPOOLS_SCHED_SUPPORTED DEIVCE_ASSOCIATED
Network Hint:0 192.168.110.0/255.255.255.240
MAC address:00:50:56:01:44:05
VLanHwTxAccel:1
VLanHwRxAccel:1
States:: 0xff -> DEVICE_PRESENT DEVICE_READY DEVICE_RUNNING DEVICE_QUEUE_OK DEVICE_LINK_OK DEVICE_PROMISC DEVICE_BROADCAST DEVICE_MULTICAST
Pseudo Device:0
Legacy vmklinux device:1
Respools sched allowed:1
Respools sched supported:1
}
[root@esx01:~] vsish -e get /net/pNics/vmnic0/rxqueues/info
rx queues info {
# queues supported:1
# filters supported:0
# active filters:0
# filters moved by load balancer:0
# of Geneve OAM filters:2
RX filter classes:Rx filter class: 0 -> No matching defined enum value found.
Rx Queue features:features: 0 -> NONE
}
[root@esx01:~] esxcfg-module -l | grep -i ipfix ipfix 12 212 nsxt-ipfix 0 68
[root@esx01:~]esxcfg-advcfg -g /Net/NetpollSwLRO Value of NetpollSwLRO is 1
[root@esx01:~] net-dvs -l
switch b1 75 16 50 72 84 37 a8-3f 1b 18 e4 55 d0 97 0d (etherswitch)
max ports: 1536
global properties:
com.vmware.common.opaqueDvs = false , propType = CONFIG
com.vmware.vrdma.uuid = 52 bc 69 d1 9b 6a 8a 52-0a d5 19 24 21 0b 09 a8 , propType = CONFIG
com.vmware.common.alias = DSwitch-CAI-Management , propType = CONFIG
com.vmware.common.uplinkPorts:
Uplink 1, Uplink 2
propType = CONFIG
... command output truncated here
[/fusion_tab][fusion_tab title=”Firmware” icon=””]
As a quick reference, this is a summary of the commands found here:
https://spillthensxt.com/are-my-esxi-host-nic-drivers-and-firmware-nsx-t-ready/
[root@esx01:~] esxcli network nic list [root@esx01:~] vmkchdev -l | grep vmnic [root@esx01:~] vmware -vl [root@esx01:~] esxcli network nic get -n vmnic0
[/fusion_tab][fusion_tab title=”-T Cmds” icon=””]
[root@esxi01:~] esxcli network ip connection list | egrep "1234|1235"
tcp 0 0 192.168.115.34:46884 192.168.120.1:1235 ESTABLISHED 267749 newreno nsx-proxy
tcp 0 0 192.168.115.34:46853 192.168.120.1:1234 ESTABLISHED 267749 newreno nsx-proxy
[root@esxi01:~] esxcli software vib list | grep nsx
nsx-adf 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-cfgagent 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-context-mux 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-cpp-libs 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-esx-datapath 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-exporter 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-host 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-idps 3.0.0.0.0-7.0.15928666 VMware VMwareCertified 2020-06-12
nsx-monitoring 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-mpa 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-nestdb 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-netopa 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-opsagent 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-platform-client 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-proto2-libs 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-proxy 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-python-gevent 1.1.0-15366959 VMware VMwareCertified 2020-06-12
nsx-python-greenlet 0.4.14-15670904 VMware VMwareCertified 2020-06-12
nsx-python-logging 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-python-protobuf 2.6.1-15670901 VMware VMwareCertified 2020-06-12
nsx-python-utils 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-sfhc 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-shared-libs 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsx-vdpi 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
nsxcli 3.0.0.0.0-7.0.15945993 VMware VMwareCertified 2020-06-12
[root@esxi01:~] nsxdp-cli vswitch instance list DvsPortset-5 (huds-nsx-lab-nvds) c4 55 d9 32 be 97 4f d6-b4 5b df 34 bd a3 7b 46 Total Ports:11776 Available:11737 Client PortID DVPortID MAC Uplink Management 184549377 00:00:00:00:00:00 n/a vmnic1 184549378 uplink-1 00:00:00:00:00:00 Shadow of vmnic1 184549379 00:50:56:53:a8:00 n/a vmk10 184549380 10 00:50:56:63:f5:24 vmnic1 vmk50 184549381 7a53b9e7-0c3c-4b68-a612-41bc3b6a03a3 00:50:56:60:0a:a4 void vdr-vdrPort 184549382 vdrPort 02:50:56:56:44:52 vmnic1
[root@esxi02:~] /usr/lib64/vmware/nsx-opsagent/bin/nsxcfg-vswitch -l NVDS Name Status Uplinks NSXToverlay up vmnic2
[root@esxi02:~] nsxdp-cli vswitch teaming ls_policy query --dvs-alias huds-nsx-lab-nvds
[root@esxi02:~] cat ./etc/vmware/nsx/appliance-info.xml
945f7a76-7cf3-4c39-bc23-6be9a5c77824
192.168.120.1
1234
true
-----BEGIN CERTIFICATE----- MIIEjDCCA3SgAwIBAgIJAIeBGxdoxqjfMA0GCSqGSIb3DQEBCwUAMIHRMQswCQYD VQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTESMBAGA1UEBxMJUGFsbyBBbHRv MRUwEwYDVQQKEwxWTXdhcmUsIEluYy4xKjAoBgkqhkiG9w0BCQEWG3NzbC1jZXJ0 aWZpY2F0ZXNAdm13YXJlLmNvbTEgMB4GA1UEAxMXVk13YXJlLU5TWC1BcHBsUHJv eHlIdWIxNDAyBgoJkiaJk/IsZAEBEyQ5NDVmN2E3Ni03Y2YzLTRjMzktYmMyMy02 YmU5YTVjNzc4MjQwHhcNMjAwNjEyMjEwNjI0WhcNMzAwNjEwMjEwNjI0WjCB0TEL MAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExEjAQBgNVBAcTCVBhbG8g QWx0bzEVMBMGA1UEChMMVk13YXJlLCBJbmMuMSowKAYJKoZIhvcNAQkBFhtzc2wt Y2VydGlmaWNhdGVzQHZtd2FyZS5jb20xIDAeBgNVBAMTF1ZNd2FyZS1OU1gtQXBw bFByb3h5SHViMTQwMgYKCZImiZPyLGQBARMkOTQ1ZjdhNzYtN2NmMy00YzM5LWJj MjMtNmJlOWE1Yzc3ODI0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA vopLXmMCWUODn1kkwbR7gQYzDEaGPjUx+ByF9QPhO5+qDJx0a3FbeX99lzrBAk+K r9YASQj6F/Kn2CHlACsnGrZ3H7/b6bGD3TsLTJjIxLaSktE1/unJv+zyWw1tc7lE XUKiUQetcWpkojHR3g+1z334maHqK+/t/DrXt5tRzqhn4FcgrXa8l+5bsB9HL7Fq v03EKzjwZXxKVIvfyGA6NgYzyWxZnfDVM2qKar0zqzvODO5y5BRtPSkqBAF7YkzL TvRRenZuRKQgE6gklh6jr4a5QjwMBkxu4ktk2l+ILEHGqoQ5yD/VcDkqKkPBqi4U F6OKDpW6GrJg/g91xlITaQIDAQABo2UwYzAJBgNVHRMEAjAAMB0GA1UdJQQWMBQG CCsGAQUFBwMBBggrBgEFBQcDAjA3BgNVHREEMDAuhixuc3gtYXBoOjk0NWY3YTc2 LTdjZjMtNGMzOS1iYzIzLTZiZTlhNWM3NzgyNDANBgkqhkiG9w0BAQsFAAOCAQEA q0MyAylmC+zXHJm/zPp+HFxnYhBKYsU3BLLS0jgXIQNWx3P4/KhMQQnaEkvpXyM5 6pRHPEKm9vtRIOCsA0SwvHF1U5aOYv4KpEDQ7XHBdNuhGFm3DnokPhXhaBh5QMo0 NHFGyOmxriJXL4dKcVJxQfWL3vVoJUnNmTV99bVwEKd5Cv7Ge5iLaYTZHJyI4/IT HGmU4LNq6yZZVr++o1ZQq2IFtzm4DLvDlFjPA7a1yP7EgaRWLSYLHmZexs1D2q6e AMvQKpaPsGRQNq1hnFkr+IZyj6hzaZmDO+6HPZUIb+KOpS7fxYtLt4+D1Somyt6y Ek9/yCDx/6Z8zJwssS2ZXw== -----END CERTIFICATE-----
MP
21b71d42-36be-5aad-1b25-7e875d4405c3
true
[root@esxi02:~] tail -f ./var/run/log/nsx-syslog.log
[root@esxi02:~] ./etc/init.d/nsx-proxy status
nsx-proxy agent service is running
[root@esxi02:~] ./etc/init.d/nsx-proxy restart
watchdog-nsx-proxy: Terminating watchdog process with PID 267729
host/vim/vmvisor/nsx-proxy
Resource pool 'host/vim/vmvisor/nsx-proxy' released.
nsx-proxy agent service is stopped
tnuuid = 6f21f757-9e39-4bd1-957a-48e024bfb1f3.
NSX Host Certificate exists
nsx-proxy agent service starts
[root@esxi02:~] nsxdp-cli bfd sessions list
Remote Local local_disc remote_disc recvd sent local_state local_diag client flaps
192.168.140.156 192.168.140.151 a585c73 48864e47 13005544 6216609 up No Diagnostic vdl2 75
192.168.140.154 192.168.140.151 7d9236fd 2e02d1c4 13005692 6217068 up No Diagnostic vdl2 19
192.168.140.155 192.168.140.151 6536fecc ecbfff8d 13003098 6217036 up No Diagnostic vdl2 35
[root@esxi01:~] net-vdr -I -l
DR Instance Information :
DR UUID: dcd02212-6eee-4943-b224-a33935b74159
DR Id: 0x00000005
Number of Lifs: 3
Number of Routes: 4
Number of Hold Pkts: 0
State: Enabled
Num unique nexthops: 1
Generation Number: 0
Edge Active: No
Pmac: 00:00:00:00:00:00
Dynamic resource pool tag: 65535
[root@esxi01:~] net-vdr -l --route dcd02212-6eee-4943-b224-a33935b74159
DR dcd02212-6eee-4943-b224-a33935b74159 Route Table
Legend: [U: Up], [G: Gateway], [C: Connected], [I: Interface]
Legend: [H: Host], [B: Blackhole], [F: Soft Flush] [!: Reject] [E: ECMP]
Destination GenMask Gateway Flags Ref UpTime HitCount Lif UUID
----------- ------- ------- ----- --- ------ -------- ---------
0.0.0.0 0.0.0.0 169.254.0.2 UG 2 330114 5871 f4f35149-3846-453d-9dfb-e7513cb67cd8
169.254.0.0 255.255.255.240 0.0.0.0 UCI 1 330114 3 f4f35149-3846-453d-9dfb-e7513cb67cd8
192.168.60.0 255.255.255.0 0.0.0.0 UCI 1 330114 11 0c4f82f3-11af-4a7d-8c8b-1b08252c3d85
192.168.70.0 255.255.255.0 0.0.0.0 UCI 1 330114 5878 95ab041d-db33-465c-9bdd-e555bc38f30a
[root@esxi01:~] net-vdr -l --lif dcd02212-6eee-4943-b224-a33935b74159
DR dcd02212-6eee-4943-b224-a33935b74159 LIF Information :
UUID: 95ab041d-db33-465c-9bdd-e555bc38f30a
Mode: Routing, Downlink
Id: Overlay:67589
Ipv4/Mask: 192.168.70.1/24
Ipv6/Mask:
Mac: 02:50:56:56:44:52
Connected Dvs: NSXToverlay
VXLAN Control Plane: Enabled
Replication Mode: 0.0.0.1
State: Enabled
Flags: 0x4388
DHCP Relay: 192.168.60.1
DAD Mode: LOOSE
RA Mode: UNKNOWN
UUID: f4f35149-3846-453d-9dfb-e7513cb67cd8
Mode: Routing-Backplane
Id: Overlay:67588
Ipv4/Mask: 169.254.0.1/28
Ipv6/Mask: fe80::50:56ff:fe56:4452/128(U)
Mac: 02:50:56:56:44:52
Connected Dvs: NSXToverlay
VXLAN Control Plane: Enabled
Replication Mode: 0.0.0.1
State: U
Flags: 0x10308
DHCP Relay: Not enabled
DAD Mode: LOOSE
RA Mode: SLAAC_DNS_THROUGH_RA(M=0, O=0)
UUID: 0c4f82f3-11af-4a7d-8c8b-1b08252c3d85
Mode: Routing, Downlink
Id: Overlay:67587
Ipv4/Mask: 192.168.60.2/24
Ipv6/Mask:
Mac: 02:50:56:56:44:52
Connected Dvs: NSXToverlay
VXLAN Control Plane: Enabled
Replication Mode: 0.0.0.1
State: Enabled
Flags: 0x388
DHCP Relay: Not enabled
DAD Mode: LOOSE
RA Mode: UNKNOWN
[/fusion_tab][fusion_tab title=”nsxcli” icon=””]
nsxcli commands can be run from admin or root accounts. As admin: esx01> get managers As root: [root@esx01:~] nsxcli -c get managers esx01> get managers - 192.168.110.17 Connected (NSX-RPC) *
esx01> get controllers Controller IP Port SSL Status Is Physical Master Session State Controller FQDN 192.168.110.17 1235 enabled connected true up NA esx01> get version VMware NSX Software, Version 2.5.1.0.0.15314289 Technical Support: http://www.vmware.com/support.html
esx01> get node-uuid 1028edaa-5e9d-4c30-8efc-cd77f42762cc
esx01> get logical-switches
Logical Switches Summary
------------------------------------------------------------
Overlay Kernel Entry ============================================================ VNI DVS name VIF num 67589 NSXToverlay 2 67587 NSXToverlay 1 67588 NSXToverlay 1
Overlay LCP Entry ============================================================ VNI Logical Switch UUID 67587 2541150b-528c-4192-9416-6f5d7eb0b16b 67588 531f0ad6-00b2-45e8-8b77-887771420d02 67589 dcc43ea0-6e47-4ff4-9bbc-9e4c6fe612c9
VLAN Backed Entry
============================================================
Logical Switch UUID VLAN ID
esx01> get logical-switch dcc43ea0-6e47-4ff4-9bbc-9e4c6fe612c9 arp-table
Logical Switch ARP Table
--------------------------------------------------
Host Kernel Entry
==================================================
IP MAC Flags
LCP Remote Entry
==================================================
IP MAC
192.168.70.100 00:50:56:96:61:ec
LCP Local Entry
==================================================
IP MAC
192.168.70.101 00:50:56:96:b9:45
esx01> get logical-switch dcc43ea0-6e47-4ff4-9bbc-9e4c6fe612c9 mac-table
Logical Switch MAC Table
---------------------------------------------------------------------------
Host Kernel Entry
===========================================================================
Inner MAC Outer MAC Outer IP Flags
00:50:56:96:61:ec 00:50:56:6f:66:20 192.168.110.182 0xd
LCP Remote Entry
===========================================================================
Inner MAC Outer MAC Outer IP
00:50:56:96:61:ec 00:50:56:6f:66:20 192.168.110.182
LCP Local Entry
===========================================================================
Inner MAC Outer MAC Outer IP
00:50:56:96:b9:45 00:50:56:6c:1c:d7 192.168.110.184
esx01> get logical-switch dcc43ea0-6e47-4ff4-9bbc-9e4c6fe612c9 vtep-table
Logical Switch VTEP Table
-----------------------------------------------------------------------------------------------
Host Kernel Entry
===============================================================================================
Label VTEP IP Segment ID Is MTEP VTEP MAC BFD count
28675 192.168.110.182 192.168.110.0 False 00:50:56:6f:66:20 1
LCP Remote Entry
===============================================================================================
Label VTEP IP Segment ID VTEP MAC DEVICE NAME
28675 192.168.110.182 192.168.110.0 00:50:56:6f:66:20 None
LCP Local Entry
===============================================================================================
Label VTEP IP Segment ID VTEP MAC DEVICE NAME
28677 192.168.110.184 192.168.110.0 00:50:56:6c:1c:d7 None
esx01> get logical-routers
Logical Routers Summary
------------------------------------------------------------
VDR UUID LIF num Route num
dcd02212-6eee-4943-b224-a33935b74159 3 4
esx01> get logical-router dcd02212-6eee-4943-b224-a33935b74159 forwarding
Logical Routers Forwarding Table
--------------------------------------------------------------------------------------------------------------
Flags Legend: [U: Up], [G: Gateway], [C: Connected], [I: Interface][H: Host], [R: Reject], [B: Blackhole], [F: Soft Flush], [E: ECMP]
Network Gateway Type Interface UUID ============================================================================================================== 0.0.0.0/0 169.254.0.2 UG f4f35149-3846-453d-9dfb-e7513cb67cd8 169.254.0.0/28 0.0.0.0 UCI f4f35149-3846-453d-9dfb-e7513cb67cd8 192.168.60.0/24 0.0.0.0 UCI 0c4f82f3-11af-4a7d-8c8b-1b08252c3d85 192.168.70.0/24 0.0.0.0 UCI 95ab041d-db33-465c-9bdd-e555bc38f30a ::/0 fe80::50:56ff:fe56:5300 UG f4f35149-3846-453d-9dfb-e7513cb67cd8 fe80::50:56ff:fe56:5300/128 :: UCI f4f35149-3846-453d-9dfb-e7513cb67cd8 fe80:408:100:0:50:56ff:fe56:4452/128 :: UCI f4f35149-3846-453d-9dfb-e7513cb67cd8 ff02:408:100::1:ff56:4452/128 :: UCI f4f35149-3846-453d-9dfb-e7513cb67cd8 ff02:408:100::1:ff56:5300/128 :: UCI f4f35149-3846-453d-9dfb-e7513cb67cd8
esx01> get logical-router dcd02212-6eee-4943-b224-a33935b74159 interfaces
Logical Router Interfaces
---------------------------------------------------------------------------
LIF UUID : 95ab041d-db33-465c-9bdd-e555bc38f30a
Mode : [b'Routing']Overlay VNI : 67589
IP/Mask : 192.168.70.1/24
Mac : 02:50:56:56:44:52
Connected DVS : NSXToverlay
Control plane enable : True
Replication Mode : 0.0.0.1
State : [b'Enabled']Flags : 0x4388
DHCP relay : 192.168.60.1
DAD-mode : ['LOOSE']RA-mode : ['UNKNOWN']
LIF UUID : f4f35149-3846-453d-9dfb-e7513cb67cd8 Mode : [b'Routing-Backplane']Overlay VNI : 67588 IP/Mask : 169.254.0.1/28; fe80::50:56ff:fe56:4452/128(U) Mac : 02:50:56:56:44:52 Connected DVS : NSXToverlay Control plane enable : True Replication Mode : 0.0.0.1 State : [b'Enabled']Flags : 0x10308 DHCP relay : Not enable DAD-mode : ['LOOSE']RA-mode : ['SLAAC_DNS_THROUGH_RA(M=0, O=0)']
LIF UUID : 0c4f82f3-11af-4a7d-8c8b-1b08252c3d85
Mode : [b’Routing’]Overlay VNI : 67587
IP/Mask : 192.168.60.2/24
Mac : 02:50:56:56:44:52
Connected DVS : NSXToverlay
Control plane enable : True
Replication Mode : 0.0.0.1
State : [b’Enabled’]Flags : 0x388
DHCP relay : Not enable
DAD-mode : [‘LOOSE’]RA-mode : [‘UNKNOWN’]
[/fusion_tab][fusion_tab title=”-T Files” icon=””]
[root@esxcna01-s1:/etc/vmware] ls -al nsx*
nsx:
total 76
drwxr-xr-x 1 root root 512 Jun 19 17:53 .
-r--r--r-T 1 root root 67 Dec 15 2019 .#appliance-info.xml
-r--r--r-T 1 root root 552 Dec 15 2019 .#controller-info.xml
-r--r--r-T 1 root root 0 Dec 15 2019 .#host-cert.pem
-r--r--r-T 1 root root 273 Dec 15 2019 .#host-cfg.xml
-r--r--r-T 1 root root 0 Dec 15 2019 .#host-privkey.pem
drwxr-xr-x 1 root root 512 Jun 24 10:43 ..
-rw-rw-r-T 1 root root 2133 Jun 24 09:11 appliance-info.xml
-rw-r--r-T 1 root root 4197 Dec 15 2019 cfgAgent.xml
-rw-rw-r-T 1 root root 3405 Jun 24 09:13 controller-info.xml
-rw-r--r-- 1 root root 1602 Jun 19 17:53 host-cert.pem
-rw-rw-r-T 1 root root 251 Jun 19 17:53 host-cfg.xml
-rw-r--r-- 1 root root 1708 Jun 19 17:53 host-privkey.pem
-rw-r--r-T 1 root root 1878 Dec 15 2019 hyperbus.xml
-rw-r--r-T 1 root root 5552 Dec 15 2019 netcpa.xml
-r--r--r-- 1 root root 181 Dec 15 2019 netopa.xml
-rw-r--r-T 1 root root 2129 Dec 15 2019 nsx-proxy.xml
-r--r--r-- 1 root root 467 Dec 15 2019 openssl-proxy.cnf
-rw-r--r-T 1 root root 467 Dec 15 2019 openssl.cnf
nsx-mpa:
total 16
drwxr-xr-x 1 root root 512 Jun 19 17:55 .
-r--r----T 1 root root 285 Dec 15 2019 .#mpaconfig.json
drwxr-xr-x 1 root root 512 Jun 24 10:43 ..
-rw-r--r-- 1 root root 0 Jun 19 17:55 .decom_state
-rw-r----- 1 root root 513 Jun 19 17:53 mpaconfig.json
nsx-opsagent:
total 40
drwxr-xr-x 1 root root 512 Jun 19 17:53 .
drwxr-xr-x 1 root root 512 Jun 24 10:43 ..
drwxr-xr-x 1 root root 512 Dec 15 2019 lldp
-rw-r--r-T 1 root root 4647 Dec 15 2019 nsx-ctxteng.xml
-rw-r--r-T 1 root root 313 Dec 15 2019 nsx-da.xml
-rw-r--r-T 1 root root 2103 Dec 15 2019 nsx-opsAgent.xml
-rw-r--r-T 1 root root 1704 Dec 15 2019 nsxa.json
-rw-r--r-T 1 root root 7323 Dec 15 2019 nsxa.json.doc
nsx-platform-client:
total 8
drwxr-xr-x 1 root root 512 Dec 15 2019 .
drwxr-xr-x 1 root root 512 Jun 24 10:43 ..
nsx-sfhc:
total 16
drwxr-xr-x 1 root root 512 Jun 19 17:55 .
drwxr-xr-x 1 root root 512 Jun 24 10:43 ..
-rw-r--r-T 1 root root 581 Dec 15 2019 sfhcConfig.ini
-rw-r--r-- 1 root root 4 Jun 24 09:13 software_status
nsxa:
total 12
drwxr-xr-x 1 root root 512 Jun 19 17:53 .
-r--r--r-T 1 root root 0 Dec 15 2019 .#host_config.bin
drwxr-xr-x 1 root root 512 Jun 24 10:43 ..
-rw-r--r-- 1 root root 576 Jun 19 19:02 host_config.bin
-rw-r--r-T 1 root root 0 Dec 15 2019 host_config_mmode.bin
[root@esxcna01-s1:/etc/vmware]
[root@esxcna01-s1:/etc/vmware] cat nsx-mpa/mpaconfig.json
{
"RmqClientType": "cvn-hv",
"RmqBrokerCluster": [
{
"BrokerFqdn": "",
"BrokerIpAddress": "192.168.110.17",
"BrokerVirtualHost": "nsx",
"BrokerIsMaster": "true",
"BrokerPort": 5671,
"BrokerSslCertThumbprint": "786f10c3c8eabbd167e4f0d96cd24181965511220561c756eaef1f06f8785d80"
}
],
"AccountName": "cvn-hv-10e905b7-19a1-47e0-9837-1ae548915967",
"SharedSecret": "8gVNsDdW4k0FCtERpfCOHyDJvmi62sc7k0RtaorqG1"
}
[root@esxcna01-s1:/etc/vmware/nsx] grep "S" controller-info.xml
192.168.110.17
1235
true
2.5.1.0.0.15314292
e8957042-d5af-40c8-9157-fe8815d91645
-----BEGIN CERTIFICATE----- MIIDOjCCAiKgAwIBAgIQbN7sFvKhQiKI48iYerIN7zANBgkqhkiG9w0BAQsFADBZ MQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExFTATBgNVBAoMDFZNd2FyZSwgSW5j LjENMAsGA1UECwwETlNCVTEXMBUGA1UEAwwObnN4LWNvbnRyb2xsZXIwHhcNMjAw MzIzMTk1NDI3WhcNMzAwMzIxMTk1NDI3WjBZMQswCQYDVQQGEwJVUzELMAkGA1UE CAwCQ0ExFTATBgNVBAoMDFZNd2FyZSwgSW5jLjENMAsGA1UECwwETlNCVTEXMBUG A1UEAwwObnN4LWNvbnRyb2xsZXIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK AoIBAQCRGHLOwoqJtvSXIGlDcid4BHFH2Zpg9RkpABLaltG0opOJ1/TLqXleua7S Es4wPPi1n3LtEbUWPoY3cX110KTKlTzMokz7tFzgvN9JqYsNCUrf9bi2SNd4Mlqa cd9MQ2yYME3bXz6WLRW4p1uDEQPl3MR4W7sO+zSDEwuJ0DpKOGN/sfs27wP4pQdb F3fGbjZ6VP13/UaaVRfZVjaJbQkccCPV+/k8RSqJbacy8higpOi+rimGfLfafJvq xgTjnZbmXjxQpX6Hh4ZwDDE8Q4/mTnSQHB6DUDzUv2V/YjO55VjF6azmMpM0eY1s wBAxWSt+o/3ws474oS0iQRVSAL5VAgMBAAEwDQYJKoZIhvcNAQELBQADggEBAHPd TWc2pmzhdPVj6m90EOqUEJ0ia/lfL5DXxn+Yepb+1Dk9Nk78joP2RwSow0ZrrU0W n7rxEpfGjVUui7+6rkuGo4OfFQkTEVQR0ZAv5vhZCsI53v3Nqjo9LEWbNaPjsC+N 9SDjftZ1uBIi6uznzFwoK2n0rhjVSPGpxwctUVaVmelDs9ycTNAjnopcnLrH8i2J noqyQqWryQWymUMcZiovE8XMlu4HB//cvEe5D+SC8HjpgK5vihL7T0AR/juqaRSD yBfs9+n9QfjZfhvo1AQuZ1jNaXXZ9YvDYiwA3ISYJqbWoH+LofUd0DZ9uuG5gQo0 skC3K/dfBHDPmlUUco4= -----END CERTIFICATE-----
10e905b7-19a1-47e0-9837-1ae548915967
[/fusion_tab][fusion_tab title=”DFW” icon=””]
From the ESXi host that you want to examine the DFW rules on, say for Guest VM named hr-web-02, that resides on that host: # summarize-dvfilter | grep -A 2 'hr-web-02' - This will return a DFW filter name that is applied to the Guest VM NIC, for example filter name: nic-35671-eth0-vmware-sfw.2
List the DFW Firewall Rules aplied to this Guest VM:
# vsipioctl getrules -f nic-35671-eth0-vmware-sfw.2
ruleset domain-c25 {
# Filter rules
rule 1015 at 1 inout protocol tcp from addrset ip-securitygroup-20 to addrset ip-securitygroup-20 port 80 accept with log;
rule 1012 at 10 inout protocol any from any to any drop with log;
List corresponding address sets:
# vsipioctl getaddrsets -f nic-35678-eth0-vmware-sfw.2
addrset ip-securitygroup-20 {
ip 172.16.10.11,
ip 172.16.10.12,
}
List counters for rules, use the -s option, to determine which rules are getting matching packets: # vsipioctl getrules -f nic-35671-eth0-vmware-sfw.2 -s - You will notice that the filter name always ends with a ".2" where the 2 references slots 2, for DFW. Once a Guest VM is added to a DFW Exclusion list you will notice that the filter is no longer applied.
[/fusion_tab][fusion_tab title=”Captures” icon=””]
A good reference to get you started: https://kb.vmware.com/s/article/2051814
In vSphere 6.5 and earlier, specify the direction of traffic using --dir 0 for inbound and --dir 1 for outbound. You can’t specify traffic going both ways at the same time. However, in vSphere 6.7 and later, you can specify the direction of traffic using --dir 0 for inbound, --dir 1 for outbound, or --dir 2 for both.
[root@esxi01:~] net-stats -l | grep <vm-name> [root@esxi01:~] pktcap-uw --switchport 33554503 --dir 0 -o switchport-outbound.pcap [root@esxi01:~] pktcap-uw --switchport 33554503 --dir 1 -o switchport-inbound.pcap [root@esxi01:~] pktcap-uw --switchport 33554503 --dir 2 -o - | tcpdump-uw -r - -nn [root@esxi01:~] pktcap-uw --switchport 33554503 --dir 2 --dstip 192.168.240.3 -o - | tcpdump-uw -r - -nn [root@esxi01:~] pktcap-uw --switchport 33554503 --dir 2 --ip 192.168.240.3 -o - | tcpdump-uw -r - -nn [root@esxi01:~] esxtop
'n' for network
'2' selects scrolls (if there are lots of Guests)
'4' to page down (to scroll through the Guests)
'TERM=xterm' (if esxtop output isn't formatted as expected) [root@esxi01:~] pktcap-uw --uplink vmnic0 --dir 0 -o - pnic-outbound.pcap [root@esxi01:~] pktcap-uw --uplink vmnic0 --dir 1 -o - pnic-inbound.pcap [root@esxi01:~] pktcap-uw --uplink vmnic0 --dir 0 -o - | tcpdump-uw -r - -nn [root@esxi01:~] pktcap-uw --uplink vmnic0 --dir 1 -o - | tcpdump-uw -r - -nn [root@esxi01:~] pktcap-uw --uplink vmnic0 --dir 2 -o - | tcpdump-uw -r - -nn | grep -i ARP
List running captures:
[root@esxi01:~] lsof |grep pktcap-uw |awk '{print $1}'| sort -u
[/fusion_tab][fusion_tab title=”Guests” icon=””]
[root@esxi01:~] net-stats -l
[root@esxi01:~] esxcli network vm list
World ID Name Num Ports Networks
-------- ---- --------- ------------------------------------
69749 VM1 1 8093a9f4-3d71-4700-af97-a6dcc291e704
71259 VM2 1 dvportgroup-18
[root@esxi1:~] esxcli network vm port list -w 69749
Port ID: 67108871
vSwitch: DvsPortset-1
Portgroup: 8093a9f4-3d71-4700-af97-a6dcc291e704
DVPort ID:
MAC Address: 00:50:56:96:98:58
IP Address: 0.0.0.0
Team Uplink: vmnic2
Uplink Port ID: 67108866
Active Filters: vmware-sfw
[root@esxi1:~] esxcli network vm port list -w 71259
Port ID: 50331656
vSwitch: DSwitch-CAI-Management
Portgroup: dvportgroup-18
DVPort ID: 19
MAC Address: 00:50:56:96:c5:31
IP Address: 0.0.0.0
Team Uplink: vmnic0
Uplink Port ID: 50331650
Active Filters: vmware-sfw, dvfilter-generic-vmware
[root@esx01:~] vim-cmd vmsvc/getallvms
[root@esx01:~] vim-cmd vmsvc/power.getstate
[root@esx01:~] vim-cmd vmsvc/power.on
[root@esxcna02-s1:~] net-stats -l
PortNum Type SubType SwitchName MACAddress ClientName
50331650 4 0 DvsPortset-0 00:50:56:01:4d:73 vmnic0
50331652 4 0 DvsPortset-0 00:50:56:01:0f:57 vmnic1
50331654 3 0 DvsPortset-0 00:50:56:01:4d:73 vmk0
50331655 3 0 DvsPortset-0 00:50:56:6f:12:ab vmk1
67108866 4 0 DvsPortset-1 00:50:56:01:0f:58 vmnic2
67108868 3 0 DvsPortset-1 00:50:56:64:19:d8 vmk10
67108869 3 0 DvsPortset-1 00:50:56:60:ba:88 vmk50
67108872 5 9 DvsPortset-1 00:50:56:96:b2:0b CentOS.eth0 . <--- Guest VM I would like to veiew statistics on
[root@esxcna02-s1:~] vsish -e get /net/portsets/DvsPortset-1/ports/67108872/stats
packet stats {
pktsTx:491
pktsTxMulticast:44
pktsTxBroadcast:56
pktsRx:732
pktsRxMulticast:108
pktsRxBroadcast:2
droppedTx:0
droppedRx:0
[/fusion_tab][fusion_tab title=”Tips” icon=””]
If you forget the esxcli command, say look for all DNS related: [root@esxi01:~] esxcli esxcli command list | grep -i dns Shutting down and starting up physical NICs: [root@esxi01:~] esxcli network nic down -n vmnic0
[root@esxi01:~] esxcli network nic up -n vmnic0
Identify which subfolder has the big file: [root@esxcna01-s1:~] cd / [root@esxcna01-s1:~] for i in *; do du -hs $i; done 336.0K VMware_bootbank_nsx-context-mux_2.5.1.0.0esx65-15314456.vib 4.0K altbootbank 129.7M bin 4.0K bootbank 496.0K bootpart.gz 17.4G dev 18.8M etc 99.8M lib 160.5M lib64 40.0K local.tgz 4.0K locker 116.0K mbr 336.0K nsx-context-mux-2.5.1.0.0esx65-15314456.zip 18.3M opt 332.6M proc 4.0K productLocker 4.0K sbin 4.0K scratch 4.0K store 760.5M tardisks 4.0K tardisks.noauto 44.0K tmp 326.9M usr 22.7M var 85.6G vmfs 12.0K vmimages 4.0K vmupgrad
Mark the syslog with text strings for log analysis:
[root@esxi01:~] esxcli system syslog mark -s="Start of Test1"
[root@esxi01:~] esxcli system syslog mark -s="End of Test1"
vSAN releated errors:
[root@esxi01:~] egrep -i '(plog|dom|lsom|vsan|rdt|cmmds)' vmkernel.log |less
[/fusion_tab][/fusion_tabs][fusion_text columns=”” column_min_width=”” column_spacing=”” rule_style=”default” rule_size=”” rule_color=”” animation_type=”” animation_direction=”left” animation_speed=”0.3″ animation_offset=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=””]
[/fusion_text][fusion_content_boxes layout=”icon-with-title” columns=”1″ heading_size=”2″ iconspin=”no” icon_align=”left” animation_direction=”left” animation_speed=”0.3″ hide_on_mobile=”small-visibility,medium-visibility,large-visibility” /][/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]