NetVanta 1638P Stops passing IP addresses

[jgnetguy]

We have 8 NetVanta 1638P switches at an installation in Torrington Ct. We inherited this installation from another company, and I believe they aren't fully configured.  
They are in various parts of a large manufacturing facility, coupled optically to the server room.

We need to power cycle the switches about every 3-4 weeks because they stop forwarding the ip addresses supplied by the dhcp servers in the server rack.

We get to a point when a device is plugged in, it never receives an IP address, so we power cycle the switch and it's ok for another few weeks.

I suspect that the company that installed them may have forgotten to configure something on these switches, and

I suspect it may be where the DNS Setup is.  The only things filled out on this page is the Host Name and there is a check mark for "Enable DNS Lookup"

however the primary and secondary dns IPs are not filled out.  Logic dictates to me that it doesn't know where to look up IP's since "enable" is checked and

there's nothing filled out.

Am I off base?  Is there something else I should look at?  I'd appreciate any input here.

Thank you.

[jayh]

DHCP isn't dependent on DNS working so that isn't your issue.

If you wanted to ping google.com from the switch itself the ping would fail because the switch wouldn't resolve google.com to an IP address.  Devices connected to the ports of the switch wouldn't have this problem because they would have DNS configured on each device either manually or by DHCP.  It wouldn't hurt anything to enter the IP addresses of your DNS servers on the switch but this is unlikely to solve the problem.

The 1638 is a layer-3 switch and as such can have a relatively complex configuration, or it could be very basic depending on how the network is designed.  From what you've provided it would be tough to guess what the issue is, but I'm not aware of any configuration option that would cause DHCP to stop working after a few weeks.

Basic troubleshooting - first gather facts.

•    When the failure occurs, is it only DHCP? If you statically configure an IP address on a device does it continue to function?  If so, look elsewhere than DHCP.
•    Are there other devices getting DHCP from the same servers not on the affected switches?  Do they continue to work?
•    What does the DHCP server log show in terms of leases?
•    Could there be a rogue DHCP server on the network?  Did someone plug in a wireless router that has DHCP turned on by default?  The devices will show an IP address on a different subnet.
• Does rebooting the DHCP server and not the switches resolve the problem?
• Anything else regarding the symptoms would be helpful.

[jgnetguy]

I will answer each question:

• When the failure occurs, is it only DHCP? If you statically configure an IP address on a device does it continue to function?  If so, look elsewhere than DHCP.  ++++++Existing devices continue to work, but if a new device comes to the effected switch, it will not get an IP address. Static configuring IP address doesn't work either.
  
  
•    Are there other devices getting DHCP from the same servers not on the affected switches?  Do they continue to work?  +++  Yes.  They continue to work.
  
  
•    What does the DHCP server log show in terms of leases?  +++ I have not checked that yet
  
  
•    Could there be a rogue DHCP server on the network?  Did someone plug in a wireless router that has DHCP turned on by default?  The devices will show an IP address on a different subnet.  +++ No, effected devices that are not getting an IP are the standard 169.etc....
  
  
• Does rebooting the DHCP server and not the switches resolve the problem? +++ No. It seems local only to the particular switch in question. They all do this at different times. 
  
  
• Anything else regarding the symptoms would be helpful.  +++The above answers is all I know at the moment.  This has been an ongoing issue.

[jayh]

The fact  that static IPs don't work sounds like some kind of ARP issue.

While it's working, capture the following on the switches:

show mac address-table

show arp

Similarly capture the arp table of your router.

Next time it fails, console in to the switches and see if anything is drastically different.

clear mac address-table dynamic may fix it if something is wedged. 

Is spanning-tree properly set up?  Any unmanaged switches that could be causing a loop?  More than one wireless access point on the same SSID and channel that could be causing a loop? 

[jgnetguy]

How do I clear the Mac Address Table Dynamic and if I do that, will it wipe any of the special configurations within the switch?

Also when it gets stuck, would it help to hit "Clear CPU Max Load"? or is that just for display status purposes?

[jgnetguy]

One more thing..  I just realized you asked if Spanning Tree was setup properly.

When I click on that I see a white screen with text at the top left that simply says: 503: server error

so I'm assuming it's not setup properly.

I am not even sure wht spanning tree is.    The company that installed and manages these 8 switches throughout our facility is Windstream.   I may have to contact them from this point.

[Anonymous]

- The commands that is referring to can be implemented in the Command Line Interface of the 1638. To answer your question regarding the "clear mac address-table dynamic" command, this will clear all the dynamic (learned) entries in the 1638's MAC address table. It will not wipe out any part of your configuration. It will just force the 1638 to repopulate the MAC address table.

The error you ran into navigating to the Spanning Tree page seems to be an issue related to the GUI display. What firmware are you running on the 1638?

Thanks,

Noor

[jayh]

jgnetguy wrote:

One more thing..  I just realized you asked if Spanning Tree was setup properly.

When I click on that I see a white screen with text at the top left that simply says: 503: server error

so I'm assuming it's not setup properly.

I am not even sure wht spanning tree is.    The company that installed and manages these 8 switches throughout our facility is Windstream.   I may have to contact them from this point.

The 503 server error is likely a bug in the GUI that may have been fixed in a later release.

Spanning-tree is a protocol that protects against loops when multiple switches are deployed in a network.  In relatively simple terms, switches learn on what port to forward a frame based on the destination MAC address, and keep this information in the MAC-address-table.  When a frame arrives for a MAC not in the table, the switch floods that frame out all ports.  If it gets a reply, the port from which the reply came is put in the MAC-address-table.

Now picture a scenario with three switches A, B, and C in a triangle.  Say that a frame comes in from switch C to switch A for an unknown MAC that isn't on the network any more, for example someone turned off their computer.  Switch A sends it to switch B, which also doesn't have a destination.  So it sends it to switch C, which sends it back to switch A, and so on.  This will render the network unusable as this lost frame continued to loop around the network churning the CPU and also flooding all of the user ports.

For redundancy, it's a good idea to have the triangle setup.  If any switch fails, the remaining switches can still communicate. 

Spanning-tree builds a tree-like structure among all of the switches and will shut down any links that would cause a loop.  In this case the link on switch C connected to switch A (for example) would be put into a blocking state.  If switch B fails, then spanning-tree will re-calculate the paths and enable the link between A and C.

There are a number of things that can be configured to determine which switch is the root, which ports take priority, etc., as well as various "flavors" of spanning-tree that speed up convergence time, deal with VLANs, etc. but that's the basic idea.

Some inexpensive switches don't do spanning-tree.  Sometimes two wireless APs can see each other and cause a loop over-the-air.  Bridging loops will cause massive problems which is why spanning-tree is important in a network of multiple switches and you need to protect against accidental loops.

You are probably going to want to get on the console of the affected switches before and during a failure to look at the topology and compare differences. 

[Anonymous]

- Do you need any further assistance regarding this issue?

Thanks,

Noor

[Anonymous]

:

I went ahead and flagged this post as "Assumed Answered." If any of the responses on this thread assisted you, please mark them as Correct or Helpful as the case may be with the applicable buttons. This will make them visible and help other members of the community find solutions more easily. If you still need assistance, we would be more than happy to continue working with you on this - just let us know in a reply.

Thanks,

Levi