Can't find what you need?


• Ask the Community
• Create a Case
Reset Search
 

 

Article

Termination of fab_vcsd and l2sysd followed by HA failover on multiple rbridges

« Go Back

Information

 
TitleTermination of fab_vcsd and l2sysd followed by HA failover on multiple rbridges
Symptoms
Termination of fab_vcsd and l2sysd followed by HA failover on multiple rbridges

There are a few possible patterns to this issue. This is one.
2020/06/22-07:46:29, [RAS-1005], 504053, SW/0 | Active | FFDC, WARNING, VDXA, Software 'assert' error detected.
2020/06/22-07:46:29, [RAS-1001], 504054, SW/0 | Active, INFO, VDX6740, First failure data capture (FFDC) event occurred.
2020/06/22-07:46:44, [HASM-1200], 504055, SW/0 | Active | FFDC, WARNING, VDX6740, Detected termination of process fab_vcsd:2295.
2020/06/22-07:46:44, [HASM-1101], 504056, SW/1 | Standby, WARNING, VDX6740, HA State out of sync.
2020/06/22-07:46:44, [HASM-1000], 504057, SW/0 | Active, CRITICAL, VDX6740, Daemon fab_vcs terminated. System initiated reload/failover for recovery.
2020/06/22-07:46:59, [HASM-1200], 504058, SW/0 | Active | FFDC, WARNING, VDX6740, Detected termination of process l2sysd:3163.

This is another.
2020/06/30-07:58:33, [RAS-1005], 1619102, SW/0 | Active | FFDC, WARNING, VDXB, Software 'assert' error detected.
2020/06/30-07:58:33, [RAS-1001], 1619103, SW/0 | Active, INFO, VDX6740, First failure data capture (FFDC) event occurred.
2020/06/30-07:58:48, [HASM-1200], 1619104, SW/0 | Active | FFDC, WARNING, VDX6740, Detected termination of process fab_vcsd:2306.
2020/06/30-07:58:48, [HASM-1101], 1619105, SW/1 | Standby, WARNING, VDX6740, HA State out of sync.
2020/06/30-07:58:49, [LOG-1000], 1619106, SW/0 | Active, INFO, VDX6740, Previous message has repeated 1 times.
2020/06/30-07:58:49, [HASM-1000], 1619107, SW/0 | Active, CRITICAL, VDX6740, Daemon fab_vcs terminated. System initiated reload/failover for recovery.
2020/06/30-07:59:06, [HASM-1200], 1619108, SW/0 | Active | FFDC, WARNING, VDX6740, Detected termination of process l2sysd:3391.
2020/06/30-07:59:13, [RAS-1001], 1619109, SW/0 | Active, INFO, VDX6740, First failure data capture (FFDC) event occurred.
Environment
  • VDX6740, VDX6740T-1G
  • NOAS 7.2.0, NOS 7.2.0cb
  • Large VCS of over 30 rbridges
  • MAC table that expands and contracts frequently, regularly grows to over 300,000 addresses
  • Frequently changing MAC table on the scale of several thousands learned and aged out each second
Cause
These process terminations happen due to a failure to release shared memory segments used for exchanging information via IPC messages between l2sysd and fab_vcsd.

These shared memory segments are like a wall of mailboxes in an office or apartment building rather than a single tank. Their usage cannot be monitored with normal NOS CLI commands such as "show process memory".
Resolution
This issue has been resolved in NOS 7.2.0e under defect NOS-67550. Software was modified to increase the number of available memory segments and to free segments no longer in use. Customers affected by this issue should upgrade immediately. There is no workaround short of upgrade.
Additional notes

Feedback

 

Was this article helpful?


   

Feedback

Please tell us how we can make this article more useful.

Characters Remaining: 255