Trouble with intermittent high ping times to VMs on ESXi Servers

[Anonymous]

First of all let me say upfront this is my first post so my apologies if it is poorly written.  We have NetVanta 1544 switches at all of our remote sites and we use them as the core at the site and we use the routing features for inter vlan routing.  At one of our sites the vm machines have started experiencing intermittent high ping times and poor performance.  We are currently running AOS version R 10.3.0.  Some other relevent facts include the two 1544 switches are connected to each other by two gig ports in a port-channel group. They run VRRP to provide redundancy on the vlan interfaces.  The ESX machines are connected to a 1534 which is connected to the 1544 with a two gig port-channel.  We run Rapid Spanning Tree to keep us loop free.  We have a NAS that is plugged into the same 1534 switch and is on the same VLAN as the VM machines that keeps low ping times even when the vm pings go nuts.  The only thing I have noticed that seems important is that when the ping times go high a process called bcmRX goes up to around 20.  If anyone out there has an idea I'm all ears.  Thanks in advance for your help!

[Anonymous]

:

Thank you for asking this question in the support community.  Have you noticed any Layer 1 type issues (errors on interfaces, bad cables, etc.)?  It appears that it could be something different between the ESX machines and the NAS, since they both connect to the same units and are on the same broadcast domain (VLAN).  On something as random as this, it is often beneficial to obtain a packet capture when the devices are experiencing latency and review the source/destination and type of traffic. 

Please, let me know if there is anything I can do to assist you.

Levi

[Anonymous · Accepted solution]

An update to the problem.  We upgraded the firmware to version R10.5.0.E and did a reboot.  We immediately noticed the ping times stayed in a normal <1 to 1 ms, but at first the bcmRX process was still high.  After a while of uptime that too has returned to normal.  I'm not sure whether it was a firmware update or reboot, but one of them seemed to fix the symptoms.  Thanks for your suggestions Levi, if the problems come back I will try the packet capture.

Best Regards

[Anonymous]

:

I went ahead and flagged this post as “Assumed Answered.”  If any of the responses on this thread assisted you, please mark them as either Correct or Helpful answers with the applicable buttons.  This will make them visible and help other members of the community find solutions more easily.  If you still need assistance, I would be more than happy to continue working with you on this - just let me know in a reply.

Levi

[Anonymous]

:

I went ahead and flagged the "Correct Answer" on this post to make it more visible and help other members of the community find solutions more easily. If you don't feel like the answer I marked was correct, feel free to come back to this post and unmark it and select another in its place with the applicable buttons.  If you still need assistance, we would be more than happy to continue working with you on this - just let us know in a reply.

Thanks,

Levi