Reset Search
 

 

Article

Subinterface for a 10G SFP in a 40G QSFP adapter in a 40G slot set to administratively down immediately after adapter inserted in slot.

« Go Back

Information

 
TitleSubinterface for a 10G SFP in a 40G QSFP adapter in a 40G slot set to administratively down immediately after adapter inserted in slot.
Symptoms
After inserting a 10G SFP in a Mellanox 40G QSFP adapter and inserting the Mellanox adapter into the sfp breakout-configured QSFP slot of a VDX6740 without connecting a cable, the 10G subinterface intended for use is suddenly set to administratively down.

After such a subinterface has been automatically set to administratively down, if manual "no shutdown" is attempted without connecting a cable, then the subinterface will be automatically set to administratively down again.

In some versions of software such as 7.3.0aa, 7.3.0a, and earlier versions, if a CBR-5047 FFDC file does not already exist under "show support", then a First failure data capture (FFDC) event is logged, and several seconds later under "show support" an FFDC file appears with filename begining with CBR-5047. If a CBR-5047 FFDC file already exists under "show support" when this issue happens, then an additional one will not be created. Either way, the 10G subinterface intended for use will still be set to administratively down.

For example, in 7.3.0a:
VDX6# conf t
Entering configuration mode terminal
VDX6(config)# interface TenGigabitEthernet 6/0/50:1
VDX6(conf-if-te-6/0/50:1)# no shutdown
2019/06/10-11:20:34, [NSM-1019], 2216, SW/0 | Active | DCE, INFO, VDX6, Interface TenGigabitEthernet 6/0/50:1 is administratively up.
2019/06/10-11:21:10, [RAS-1001], 2217, SW/0 | Active, INFO, VDX6740T, First failure data capture (FFDC) event occurred.
2019/06/10-11:21:10, [NSM-1020], 2218, SW/0 | Active | DCE, INFO, VDX6, Interface TenGigabitEthernet 6/0/50:1 is administratively down.
VDX6(conf-if-te-6/0/50:1)# end
VDX6# show suppor
Rbridge-id: 6
#SW/0:
Mon Jun 10 11:21:34 2019 /core_files/ffdc_data/CBR-5047-2019-06-10-112134.ffdc
#SW/1:
No core or FFDC data files found!

In some versions of software such as NOS 7.4.0 and possibly later versions, no FFDC file will be generated but the 10G subinterface will still be set to administratively down. A CBR-1042 Port Hard Fault may be logged instead of the CBR-5047 FFDC.

For example, in 7.4.0:
VDX6(config)# interface TenGigabitEthernet 6/0/50:1
VDX6(conf-if-te-6/0/50:1)# no shutdown
2019/06/10-12:19:03, [NSM-1019], 2380, SW/0 | Active | DCE, INFO, VDX6, Interface TenGigabitEthernet 6/0/50:1 is administratively up.
2019/06/10-12:19:05, [CBR-1042], 2381, SW/0 | Active, ERROR, VDX6, UPSM [OID 0x43014020] (113) (UP04): Port hard fault, reason code = 15. Port is now offline. No shut to bring back port online
2019/06/10-12:19:06, [NSM-1020], 2382, SW/0 | Active | DCE, INFO, VDX6, Interface TenGigabitEthernet 6/0/50:1 is administratively down.

Usually when this issue is observed, it happens when no cable is connected to the SFP. It can also happen with a cable connected if the other end of the cable is physically disconnected.

If a 10G SFP in a Mellanox adapter is used on an ISL interface, in rare cases after a reload or firmware download, if each VDX on each end of the ISL happens to boot up at slightly different times, this issue can happen.

If a cable is connected to a 10G SFP in a Mellanox adapter and the other end of the link is administratively down but still emitting an optical signal, then this issue cannot happen. The subinterface will show as line protocol down but not administratively down.
Any version of this issue with any type of SFP or cable will not happen as long as both ends of the cable are connected and linked up.

This issue was reported with a 10G SFP in a Mellanox adapter with no cable connected. A similar issue has also been observed with 10G DAC cables, also known as Twinax cables, if the SFP at the other end of the cable is not plugged into anything. It can also happen if the other end of the DAC cable is not yet capable of linking up due to the device at the other end being in the middle of a reload. With 10G SFPs in Mellanox adapters, this issue has been observed on sfp breakout-configured subinterfaces of 40G interfaces. With DAC cables, this issue has been observed on generic 10G SFP slot interfaces.

A similar issue has also been observed with
40G QSFP to 4x10G SFP pigtail cables when a server using one of the 10G SFPs was rebooted.
Environment
  • VDX6740, VDX6740T, or VDX6740T-1G
  • 10G SFP in a Mellanox 40G QSFP slot adapter in a 40G QSFP slot for an interface configured with sfp breakout
  • 10G DAC (Twinax) cable in any slot
  • 40G QSFP to 4x10G SFP pigtail cable
Cause
When no cable is connected to a 10G SFP in a Mellanox adapter, even when there is no hardware fault, random noise internal to the VDX and the SFP can result in too many interrupt request hardware signals generated within the VDX. Interfaces that generate too many interrupts are automatically set to administratively down to protect the system. When similar issues are observed with other types of optics and cables, the same cause applies.
Resolution
When a system detects too many interrupts from an interface it is expected behavior and a good thing for the system to automatically set the interface to administratively down. This particular issue only happens when a cable is not physically connected to the SFP or is connected to the SFP but not connected at the other end. An affected 10G subinterface with no cable connected can safely be left administratively down. After connecting a cable to the SFP and connecting the other end of the cable to another device, applying "no shutdown" to the settings of the subinterface will allow it to safely link up. Similar issues with other types of optics and cables can also be resolved by making sure both ends of cables are physically connected and applying "no shutdown" to the settings of the interface or subinterface.

When using NOS 7.4.0, after connecting a cable and executing "no shutdown", in rare instances CBR-1042 Port hard fault and administratively down may reoccur. This condition becomes more likely to reoccur after repeatedly removing and inserting the SFP or repeatedly removing and inserting the Mellanox adapter. When this happens even though a cable is connected, in order to recover it is necessary to reload the unit.
Additional notes

Feedback

 

Was this article helpful?


   

Feedback

Please tell us how we can make this article more useful.

Characters Remaining: 255