Can't find what you need?


• Ask the Community
• Create a Case
Reset Search
 

 

Article

After performing ISSU from 6.0.1 or 6.0.1a to a 6.0.2x release, a 1G copper SFP interface goes down and does not come back up.

« Go Back

Information

 
TitleAfter performing ISSU from 6.0.1 or 6.0.1a to a 6.0.2x release, a 1G copper SFP interface goes down and does not come back up.
Symptoms
After performing ISSU from 6.0.1 or 6.0.1a to a 6.0.2x release or ISSU from 6.0.1 to 6.0.1a, a 1G copper SFP interface goes down and does not come back up.
Environment
  • VDX6740
  • 1G copper SFP
  • ISSU from NOS 6.0.1 or 6.0.1a to any 6.0.2x release
  • ISSU from 6.0.1 to 6.0.1a.
Cause
Software versions NOS 6.0.2 and later store SFP media information in warm memory that can be accessed by either active partition before or after an ISSU upgrade. This memory structure is not created during an ISSU upgrade. It is only ever created after a coldboot upgrade, a full "reload system" reboot, or a cold reboot.

After an ISSU upgrade from NOS 6.0.1 or 6.0.1a to any 6.0.2x release, or from 6.0.1 to 6.0.1a, when an interface flaps, the software tries to access SFP media information in this warm memory location that has not been created. Software fails to obtain the SFP media information it needs to bring an interface back up, so the interface stays down.


Implementation of this warn memory structure was started in 6.0.1a and completed in 6.0.2. As such ISSU upgrades from 6.0.1 to 6.0.1a and ISSU upgrades from either 6.0.1 or 6.0.1a to any 6.0.2x release are all potentially affected.
Resolution
After this issue happens, as a temporary workaround when the VDX interface fails to come back up, applying "shutdown" and "no shutdown" to the VDX interface settings will allow it to come back up temporarily.

This is not a permanent solution. The same issue will reoccur if the next upgrade is also ISSU. The issue can also theoretically reoccur if there is a HA failover after a triggering ISSU. For a permanent solution after the issue has happened, either a coldboot upgrade or "reload system" must be performed.

To avoid this issue completely, ISSU must not be used when upgrading from NOS 6.0.1 or NOS 6.0.1a to any NOS 6.0.2x release or when upgrading from 6.0.1 to 6.0.1a. Coldboot must be used instead.

This issue is acknowledged and documented in release notes with the following note that appears in 6.0.2x release notes such as
nos6.0.2_releasenotes_v1.0.pdf under the section "6.0.2 Caveats":

"After ISSU upgrade from 6.0.1 to 6.0.1a, if the peer port (40G/Breakout/1G) goes down for any reason then the local port may go down and not come up. To recover, it is required to perform shut/no shut on the local port."

At the time that caveat was written, the possibility of consecutive ISSU without reloads or HA failover after ISSU was not considered. With those possibilities, the only safe method of preventing this issue from reoccurring is to avoid ISSU and use coldboot when NOS 6.0.1 or 6.0.1a is involved.
Additional notes

Feedback

 

Was this article helpful?


   

Feedback

Please tell us how we can make this article more useful.

Characters Remaining: 255