what are some common causes for cpu utilization to reach 100% on NV1544

[service1icn]

Pings sent to NV1544 steady at 22ms then spike to 600-800ms during low traffic. CPU utilization also spikes to 100% at the same time.

[jayh]

The command "show process cpu" should give some detail as to what is causing it.

Is the switch reachable from the Internet either directly or by port translation behind a NAT? One cause of high CPU is bots scanning or trying to brute-force SSH passwords. Filtering management to trusted networks if the device is accessible from the Internet will fix this as well as prevent the bad guys from succeeding in guessing your credentials.

Also, traffic to a switch or router is handled at a much lower priority than traffic through it, so unless it's an extreme case high CPU typically doesn't impact performance.

[Anonymous]

High CPU can be from many things.  The enable command "sh proc cpu" will show you what processes are running and what is causing the CPU to run high.

Using the web interface and any other management access will show up as "PC Config" and it is normal for this to spike to 100% when you are pulling up a new or updating a web page.

You asked for common reasons -

- SNMP requests that are not in the standard or ADTRAN MIBs.

- Managment interface(s) being used

- Multicast or broadcast network traffic

- spanning tree is reconverging

All Management access ( including ping of an interface ) is the lowest priority to the CPU, so dropped, missing, or even unresponsive pings is not unusual with high broadcast, multicast and even other management traffic.

With network design, you should always be able to access the switch.

Your network is running well if the average load on the CPU is lower than 30% most of the time., as shown by the enable command  "sh proc cpu his" .

Hope this information helps -  Jay's comments are all true, here also 🙂

[service1icn]

Ah, the low priority on traffic to the switch vs through the switch accounts for steady ping response from the other switches and servers?

No access from internet. Logs do not show anybody knocking at the management interface.

[service1icn]

show proc cpu his

#: Average load % per interval

@: Maximum load % per interval

Most current interval starts on the left.

Previous 1 minute system load:

100     # ##                                                   

90     # ##                                                   

80     # ##                                                   

70     # ##                                                   

60    #####                                                   

50    #####                                                   

40  #######                                                   

30  #######                                                   

20  #######                    ##                             

10  #######################################  ##################

  0 +++++++++1+++++++++2+++++++++3+++++++++4+++++++++5+++++++++6

             0         0         0         0         0         0

                              seconds

Previous 1 hour system load:

100 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

90 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

80 #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

70 #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

60 #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

50 #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

40 #@@@@@@@#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

30 #@@@@@@@#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

20 ######################@######################@##############

10 ############################################################

  0 +++++++++1+++++++++2+++++++++3+++++++++4+++++++++5+++++++++6

             0         0         0         0         0         0

                              minutes

Previous 72 hours system load:

100 @@@@@@@@@@@@@@@@@

90 @@@@@@@@@@@@@@@@@

80 @@@@@@@@@@@@@@@@@

70 @@@@@@@@@@@@@@@@@

60 @@@@@@@@@@@@@@@@@

50 @@@@@@@@@@@@@@@@@

40 @@@@@@@@@@@@@@@@@

30 @@@@@@@@@@@@@@@@@

20 ###############@#

10 #################

  0 +++++++++1+++++++++2+++++++++3+++++++++4+++++++++5+++++++++6+++++++++7++

             0         0         0         0         0         0         0 

                                     hours

Can you help interpret this output?

[Anonymous]

This shows the CPU is running at 10 to 20 % average and something spiked it in the last minute.  "sh proc cpu" will show processes.

[service1icn]

show proc cpu

System load: 1sec:6.89%  1min:14.83%  5min:13.94%  Min: 0.00%  Max: 100.00%

Context switch load: 0.05%

                                      Invoked  Exec Time    Runtime    Load %%

Task Id    Task Name        PRI STA   (count)     (usec)     (usec)     (1sec)

1          Idle               0 W    34709142       1745     934672      93.47

3          PC Config          7 S     5091181       4952      13596       1.36

4          PacketRouting     44 W     2654130        300       5101       0.51

5          Timer             46 W     6242199         15       1232       0.12

6          Thread Pool        4 W         246        239          0       0.00

7          FrontPanel        43 W     1253441         63       1058       0.11

8          con0              46 W      254434         10         39       0.00

9          Flash Maintena~    4 W        3809       2050          0       0.00

10         ICP Session        8 W        6561         10          0       0.00

11         RSTP              43 W     1405139        135       2640       0.26

12         RSTP-BG           42 W         466       9418          0       0.00

13         MLD Thread         6 W           0      41245          0       0.00

14         RouteTableTick     6 R      102462         77         77       0.01

15         RouteTableTick     6 R      100792         73         73       0.01

16         IP Events         27 W      119236         64         68       0.01

17         tcptimer          25 W      102071         12         68       0.01

18         tcpinp            25 W       98184        211       1554       0.16

19         tcpout            25 W      166667         28       1775       0.18

20         eth0Switch        46 W    27884263        126       3887       0.39

21         Stacking           9 W       68896         55         55       0.01

23         bcmDPC            40 W           0        380          0       0.00

24         BcmSDK Diag Sh~    5 R      591742          5         44       0.00

26         bcmL2X.0          40 W      438890       5942      18079       1.81

27         bcmCNTR.0         40 W     2276373        592       8440       0.84

28         bcmTX             40 W      378057         36        314       0.03

29         bcmXGS3AsyncTX    40 W           0        361          0       0.00

30         bcmLINK.0         40 W    17422336         16       3680       0.37

31         bcmRX             11 W     1411449         55       2460       0.25

32         SwitchQ           18 W    10214539         30       3796       0.38

33         Igmp Reswitch      6 W          93         39          0       0.00

34         DhcpSnooping      35 W           0        162          0       0.00

35         SnmpThread         6 W     2947633         14        531       0.05

36         DnsClient         19 W       32789         80         80       0.01

37         DnsProxy          19 W       12827         19          0       0.00

38         DnsTable          19 W       12783          6          0       0.00

39         WWW               22 W     1926226        112          0       0.00

40         FTPServer List~    5 W          13          7          0       0.00

41         SMTP Client       19 W           0        258          0       0.00

42         SNTP Client       22 W           0        385          0       0.00

43         CLIInjectQ         6 W           0       1926          0       0.00

45         OSPF               6 W           1         14          0       0.00

46         RipOut             6 W       61630         16         15       0.00

47         RipIn              6 W           0      42246          0       0.00

48         Fan Control        8 W      416297         24        187       0.02

50         AUTOLINKQ          4 W         785         61          0       0.00

51         HttpClientQ        6 W           0        381          0       0.00

52         UDP In            42 W       12164         95          0       0.00

53         UDP Relay         22 W           1        179          0       0.00

54         DHCP Server       34 W           0        342          0       0.00

Not much going on here.

[Anonymous]

Looking good here.  This is a snapshot in time, so when the CPU is higher, you will get different results.

[service1icn]

jroad I am seeing spikes in the WWW row up too 80%.

Can you give any detail as to what the WWW process is?

[service1icn]

Is there a document that would give insight to the process name and what it is doing?

[james-in-ca]

I had this happen in my 3120. Help here alerted me to IP FLow/Monitoring causing this.

3120 - High CPU Utilization and slowness

[jayh]

That's the webserver for GUI management. Typical to see spikes while you're accessing or configuring the device via the GUI, especially if using https.

Note that on boot, the CPU will go to or near to 100% while loading files, initializing, etc. so the box will show that CPU was at one time 100% max. You can reset this with the command "clear processes cpu max".