Reset Search
 

 

Article

When attempting to replace an rbridge in a logical chassis VCS, the replacement VDX reboots while trying to join the VCS.

« Go Back

Information

 
TitleWhen attempting to replace an rbridge in a logical chassis VCS, the replacement VDX reboots while trying to join the VCS.
Symptoms
When attempting to replace an rbridge in a logical chassis VCS, the replacement VDX reboots while trying to join the VCS.

Each time it reboots, the replacement displays a similar series of messages on its serial console.
2019/09/05-21:27:46, [HASM-1200], 5190, FFDC, WARNING, VDX5, Detected termination of process nsm:2885.
2019/09/05-21:27:46, [RAS-1001], 5191,, INFO, VDX5, First failure data capture (FFDC) event occurred.
2019/09/05-21:27:47, [HASM-1000], 5192,, CRITICAL, VDX5, Component nsm terminated. System initiated reload/failover for recovery.
2019/09/05-21:30:33, [HASM-1004], 5196,, INFO, VDX5, Processor reloaded - Software Fault: KSWD.
2019/09/05-21:40:14, [HASM-1200], 5490, FFDC, WARNING, VDX5, Detected termination of process nsm:2947.
2019/09/05-21:40:14, [RAS-1001], 5491,, INFO, VDX5, First failure data capture (FFDC) event occurred.
2019/09/05-21:40:14, [HASM-1000], 5492,, CRITICAL, VDX5, Component nsm terminated. System initiated reload/failover for recovery.
2019/09/05-21:43:20, [HASM-1004], 5496,, INFO, VDX5, Processor reloaded - Software Fault: KSWD.

Just before each reboot, the replacement VDX displays on its serial console multiple raslog messages showing that null interfaces are being added to VLANs.
2019/09/05-21:27:46, [NSM-1017], 5187, DCE, INFO, VDX5,  Interface (null) is added on interface vlan 12-13,86,114-115,119-122,135,145,207,814,817,866-867,1200,1526,1560,1585,1632,1636,1645-1647,1653,1677,1681,1689.
2019/09/05-21:27:46, [NSM-1017], 5188, DCE, INFO, VDX5,  Interface (null) is added on interface 1712,1719,1736,1749,1752-1753,1755,1757,1763,1767,1769,1771,1775,1794,1806-1814,1818-1819,1835,1837,1841,1850,1860-1863.
2019/09/05-21:27:46, [NSM-1017], 5189, DCE, INFO, VDX5,  Interface (null) is added on interface 1865,1873,1875-1877.
Environment
  • VDX6740
  • NOS 4.1.0a
  • VCS running config contains one or more switchport trunk port-channel interfaces with "switchport trunk allowed vlan add" lines with more than 256 bytes of VLAN listings
Cause
This issue is due to a known defect found and fixed during development of NOS 4.1.1. The fix is included in NOS 4.1.1, NOS 4.0.2, NOS 5.0.0, and later versions. It was found during internal development testing, so it is not listed in release notes.

The synopsis of the defect is "NSM crashes when vlan string or ctag string length goes beyond 256 characters." What this means is that if a port channel switchport trunk interface has a "switchport trunk allowed vlan add" line with more than 256 bytes of VLAN listings, when a VDX tries to read those settings NSM will crash and the VDX will reboot itself.

The following is an example of port-channel settings that will trigger this defect's reboot when a VDX tries to read them. It contains 372 bytes of VLAN listings on one line.
interface Port-channel 6
 vlag ignore-split
 switchport
 switchport mode trunk
 switchport trunk allowed vlan add 12-13,86,114-115,119-122,135,145,207,814,817,866-867,1200,1526,1560,1585,1632,1636,1645-1647,1653,1677,1681,1689,1712,1719,1736,1749,1752-1753,1755,1757,1763,1767,1769,1771,1775,1794,1806-1814,1818-1819,1835,1837,1841,1850,1860-1863,1865,1873,1875-1877,1884,1887-1888,1891-1893,1939,2300-2303,2306,2310,2312,2315-2318,2320,2324,2338-2339,2341,2344-2346,2367,2369,2382,2449
 switchport trunk tag native-vlan
 bpdu-drop enable all
 spanning-tree shutdown
 no shutdown

One characteristic of this defect is that the NSM crash and VDX reboot are only triggered when the line of VLAN listings is read all at once. It is possible to create these settings in a logical chassis VCS running-config by adding VLANs in segments less than 256 bytes. For example:
switchport trunk allowed vlan add 12-13,86,114-115,119-122,135,145,207,814,817,866-867,1200,1526,1560,1585,1632,1636,1645-1647,1653,1677,1681,1689,1712,1719,1736,1749,1752-1753,1755,1757,1763,1767,1769,1771,1775,1794
switchport trunk allowed vlan add 1806-1814,1818-1819,1835,1837,1841,1850,1860-1863,1865,1873,1875-1877,1884,1887-1888,1891-1893,1939,2300-2303,2306,2310,2312,2315-2318,2320,2324,2338-2339,2341,2344-2346,2367,2369,2382,2449

When you use "vcs replace" to add a VDX to a VCS that already has 372 bytes of VLANs on a single line, the newly added VDX tries to read that single line all at once. NSM on the newly added VDX crashes, and the VDX reboots.
Resolution
NOS 4.1.0a is no longer an officially supported software version. A document on Extreme Networks Software Lifecycle Policy and Key Dates can be found on our page of End of Sale and End of Support documentation and announcements. If this issue is encountered in NOS 4.1.0a, for future stability and supportability it is critical to upgrade to a supported version of software.

As a temporary workaround, it is possible to add a VDX to such a VCS by temporarily reducing the length of "switchport trunk allowed vlan add" lines during the "vcs replace" process.
  1. On the VCS principal switch CLI, in the running config add "shutdown" to any port-channel interfaces that have "switchport trunk allowed vlan add" lines with more than 256 bytes of VLAN listings.
  2. On the VCS principal switch CLI, remove long "switchport trunk allowed vlan add" lines from shutdown port-channel interfaces using multiple runs of "switchport trunk allowed vlan remove" with less than 256 bytes of VLANs each. For example:
    switchport trunk allowed vlan remove 12-13,86,114-115,119-122,135,145,207,814,817,866-867,1200,1526,1560,1585,1632,1636,1645-1647,1653,1677,1681,1689,1712,1719,1736,1749,1752-1753,1755,1757,1763,1767,1769,1771,1775,1794
    switchport trunk allowed vlan remove 1806-1814,1818-1819,1835,1837,1841,1850,1860-1863,1865,1873,1875-1877,1884,1887-1888,1891-1893,1939,2300-2303,2306,2310,2312,2315-2318,2320,2324,2338-2339,2341,2344-2346,2367,2369,2382,2449
  3. Use "vcs replace" to add the new VDX to the VCS.
  4. Wait until "show vcs" on the principal shows the new VDX VCS Status and Fabric Status as both online.
  5. On the VCS principal switch CLI, re-add long "switchport trunk allowed vlan add" lines to shutdown port-channel interfaces using multiple runs of "switchport trunk allowed vlan add" with less than 256 bytes of VLANs each. For example:
    switchport trunk allowed vlan add 12-13,86,114-115,119-122,135,145,207,814,817,866-867,1200,1526,1560,1585,1632,1636,1645-1647,1653,1677,1681,1689,1712,1719,1736,1749,1752-1753,1755,1757,1763,1767,1769,1771,1775,1794
    switchport trunk allowed vlan add 1806-1814,1818-1819,1835,1837,1841,1850,1860-1863,1865,1873,1875-1877,1884,1887-1888,1891-1893,1939,2300-2303,2306,2310,2312,2315-2318,2320,2324,2338-2339,2341,2344-2346,2367,2369,2382,2449
  6. On the VCS principal switch CLI, in the running config apply "no shutdown" to any port-channel interfaces that had been shutdown.
Additional notes

Feedback

 

Was this article helpful?


   

Feedback

Please tell us how we can make this article more useful.

Characters Remaining: 255