Reset Search
 

 

Article

All of the interfaces for a given linecard are suddenly recorded in 'show logging raslog' with 'is protocol down', 'is link down', and LACP 'MUX state transition'

« Go Back

Information

 
TitleAll of the interfaces for a given linecard are suddenly recorded in 'show logging raslog' with 'is protocol down', 'is link down', and LACP 'MUX state transition'
Symptoms
All of the interfaces for a given linecard are suddenly recorded in 'show logging raslog' with 'is protocol down', 'is link down', and either LACP 'MUX state transition "aggregate" to "not aggregate"(reason: link down) ' or LACP 'MUX state transition "aggregate" to "not aggregate"(reason: timeout) '.
YYYY/MM/DD-06:06:15, [NSM-1002], 6659, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/1 IP_SLX9850n1_01 is protocol down.
YYYY/MM/DD-06:06:15, [NSM-1003], 6660, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/1 IP_SLX9850n1_01 is link down. 
YYYY/MM/DD-06:06:15, [LACP-1006], 6661, M1 | Active | DCE, INFO, SLX9850n2,  LACP Ethernet 2/1: MUX state transition "aggregate" to "not aggregate"(reason: link down) 
YYYY/MM/DD-06:06:16, [NSM-1002], 6662, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/2 IP_SLX9850n1_02 is protocol down.
YYYY/MM/DD-06:06:16, [NSM-1003], 6663, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/2 IP_SLX9850n1_02 is link down. 
YYYY/MM/DD-06:06:16, [LACP-1006], 6664, M1 | Active | DCE, INFO, SLX9850n2,  LACP Ethernet 2/2: MUX state transition "aggregate" to "not aggregate"(reason: link down) 
YYYY/MM/DD-06:06:16, [NSM-1002], 6665, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/3 IP_SLX9850n1_03 is protocol down.
YYYY/MM/DD-06:06:16, [NSM-1003], 6666, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/3 IP_SLX9850n1_03 is link down. 
YYYY/MM/DD-06:06:16, [LACP-1006], 6667, M1 | Active | DCE, INFO, SLX9850n2,  LACP Ethernet 2/3: MUX state transition "aggregate" to "not aggregate"(reason: link down) 
YYYY/MM/DD-06:06:17, [NSM-1002], 6668, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/4 IP_SLX9850n1_04 is protocol down.
YYYY/MM/DD-06:06:17, [NSM-1003], 6669, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/4 IP_SLX9850n1_04 is link down. 
YYYY/MM/DD-06:06:17, [LACP-1006], 6670, M1 | Active | DCE, INFO, SLX9850n2,  LACP Ethernet 2/4: MUX state transition "aggregate" to "not aggregate"(reason: link down) 
YYYY/MM/DD-06:06:17, [LACP-1006], 6671, M1 | Active | DCE, INFO, SLX9850n2,  LACP Ethernet 2/5: MUX state transition "aggregate" to "not aggregate"(reason: timeout) 
YYYY/MM/DD-06:06:17, [LACP-1006], 6672, M1 | Active | DCE, INFO, SLX9850n2,  LACP Ethernet 2/6: MUX state transition "aggregate" to "not aggregate"(reason: timeout) 
YYYY/MM/DD-06:06:17, [LACP-1006], 6673, M1 | Active | DCE, INFO, SLX9850n2,  LACP Ethernet 2/8: MUX state transition "aggregate" to "not aggregate"(reason: timeout) 
YYYY/MM/DD-06:06:17, [LACP-1006], 6674, M1 | Active | DCE, INFO, SLX9850n2,  LACP Ethernet 2/9: MUX state transition "aggregate" to "not aggregate"(reason: timeout) 
YYYY/MM/DD-06:06:17, [NSM-1003], 6675, M1 | Active | DCE, INFO, SLX9850n2,  interface Port-channel 2 lag-to-kam-1 is link down. 
YYYY/MM/DD-06:06:17, [NSM-1002], 6676, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/5 IP_SLX9850n1_05 is protocol down.
YYYY/MM/DD-06:06:17, [NSM-1003], 6677, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/5 IP_SLX9850n1_05 is link down. 
YYYY/MM/DD-06:06:18, [NSM-1002], 6678, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/6 IP_SLX9850n1_06 is protocol down.
YYYY/MM/DD-06:06:18, [NSM-1003], 6679, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/6 IP_SLX9850n1_06 is link down. 
YYYY/MM/DD-06:06:19, [NSM-1002], 6680, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/8 IP_SLX9850n1_07 is protocol down.
YYYY/MM/DD-06:06:19, [NSM-1003], 6681, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/8 IP_SLX9850n1_07 is link down. 
YYYY/MM/DD-06:06:21, [NSM-1002], 6682, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/9 IP_SLX9850n1_08 is protocol down.
YYYY/MM/DD-06:06:21, [NSM-1003], 6683, M1 | Active | DCE, INFO, SLX9850n2,  interface Ethernet 2/9 IP_SLX9850n1_08 is link down.
'show support' shows no core dumps or FFDC files on the linecard, and 'show chassis' shows no change in Time Awake.

On connected devices, LACP timeouts, link downs, and local faults are reported for all connected interfaces.
YYYY/MM/DD-06:06:15, [NSM-1002], 23788, M1 | Active | DCE, INFO, SLX9850n1,  interface Ethernet 2/4 IP_SLX9850n2_04 is protocol down.
YYYY/MM/DD-06:06:15, [NSM-1003], 23789, M1 | Active | DCE, INFO, SLX9850n1,  interface Ethernet 2/4 IP_SLX9850n2_04 is link down. 
YYYY/MM/DD-06:06:15, [HSL-1021], 23790, L2/0 | Active, ERROR, SLX9850n1, interface Ethernet 2/5 IP_SLX9850n2_05 detected local fault.
YYYY/MM/DD-06:06:15, [NSM-1002], 23791, M1 | Active | DCE, INFO, SLX9850n1,  interface Ethernet 2/5 IP_SLX9850n2_05 is protocol down.
YYYY/MM/DD-06:06:15, [LACP-1006], 23792, M1 | Active | DCE, INFO, SLX9850n1,  LACP Ethernet 2/4: MUX state transition "aggregate" to "not aggregate"(reason: link down) 
YYYY/MM/DD-06:06:15, [NSM-1003], 23793, M1 | Active | DCE, INFO, SLX9850n1,  interface Ethernet 2/5 IP_SLX9850n2_05 is link down. 
YYYY/MM/DD-06:06:15, [HSL-1021], 23794, L2/0 | Active, ERROR, SLX9850n1, interface Ethernet 2/6 IP_SLX9850n2_06 detected local fault.
YYYY/MM/DD-06:06:15, [LACP-1006], 23795, M1 | Active | DCE, INFO, SLX9850n1,  LACP Ethernet 2/5: MUX state transition "aggregate" to "not aggregate"(reason: link down)

 
Environment
  • SLX9850
Cause
In rare instances, SLX linecards can experience kernel panics sudden and severe enough that recording in 'show support' and recovery through automatic reboot of the linecard are prevented. In the scenario above, the linecard that did not record any local faults, the linecard that recorded only LACP, protocol down, and link down messages is the one that experienced kernel panic.
Resolution
If possible, try to collect a 'copy support' before attempting recovery. The resulting supportsave may contain data from the affected linecard that may allow for further investigation.

Recovery can be achieved through '
power-off linecard <SLOT_NUMBER>' and 'power-on linecard <SLOT_NUMBER>' without swapping out the linecard.

The possibility of a linecard kernel panic severe and sudden enough to prevent both data collection and automatic recovery should be resolved in 18r.1.00c.
Additional notes

Feedback

 

Was this article helpful?


   

Feedback

Please tell us how we can make this article more useful.

Characters Remaining: 255