This issue is due to a known defect found and fixed during development of NOS 4.1.1. The fix is included in NOS 4.1.1, NOS 4.0.2, NOS 5.0.0, and later versions. It was found during internal development testing, so it is not listed in release notes.
The synopsis of the defect is "NSM crashes when vlan string or ctag string length goes beyond 256 characters." What this means is that if a port channel switchport trunk interface has a "switchport trunk allowed vlan add" line with more than 256 bytes of VLAN listings, when a VDX tries to read those settings NSM will crash and the VDX will reboot itself.
The following is an example of port-channel settings that will trigger this defect's reboot when a VDX tries to read them. It contains 372 bytes of VLAN listings on one line.
interface Port-channel 6
switchport mode trunk
switchport trunk allowed vlan add 12-13,86,114-115,119-122,135,145,207,814,817,866-867,1200,1526,1560,1585,1632,1636,1645-1647,1653,1677,1681,1689,1712,1719,1736,1749,1752-1753,1755,1757,1763,1767,1769,1771,1775,1794,1806-1814,1818-1819,1835,1837,1841,1850,1860-1863,1865,1873,1875-1877,1884,1887-1888,1891-1893,1939,2300-2303,2306,2310,2312,2315-2318,2320,2324,2338-2339,2341,2344-2346,2367,2369,2382,2449
switchport trunk tag native-vlan
bpdu-drop enable all
One characteristic of this defect is that the NSM crash and VDX reboot are only triggered when the line of VLAN listings is read all at once. It is possible to create these settings in a logical chassis VCS running-config by adding VLANs in segments less than 256 bytes. For example:
switchport trunk allowed vlan add 12-13,86,114-115,119-122,135,145,207,814,817,866-867,1200,1526,1560,1585,1632,1636,1645-1647,1653,1677,1681,1689,1712,1719,1736,1749,1752-1753,1755,1757,1763,1767,1769,1771,1775,1794
switchport trunk allowed vlan add 1806-1814,1818-1819,1835,1837,1841,1850,1860-1863,1865,1873,1875-1877,1884,1887-1888,1891-1893,1939,2300-2303,2306,2310,2312,2315-2318,2320,2324,2338-2339,2341,2344-2346,2367,2369,2382,2449
When you use "vcs replace" to add a VDX to a VCS that already has 372 bytes of VLANs on a single line, the newly added VDX tries to read that single line all at once. NSM on the newly added VDX crashes, and the VDX reboots.