Can't find what you need?


• Ask the Community
• Create a Case
Reset Search
 

 

Article

A VDX6740T-1G suddenly rebooted in prooduction and failed to boot afterwards.

« Go Back

Information

 
TitleA VDX6740T-1G suddenly rebooted in prooduction and failed to boot afterwards.
Symptoms
A VDX6740T-1G suddenly rebooted in prooduction and failed to boot afterwards. When moved to a separate location and powered up, it displayed the messages below, among others, and failed to boot up.
 
Starting Dcmd ... Fri Mar 22 13:07:43 JST 2019
server starting
Restoring DCM databasetp_register, got none fss tp handle=10000001h
2019/03/22-13:07:46, [HASM-1004], 26029,  INFO, VDX6740T-1G, Processor reloaded - Software Fault: KSWD.
.......................................................................................pg_ctl: PID file "/etc/fabos/Dcmd/WaveDatabase/postmaster.pid" does not exist
Is server running?
Dcmd Database is corrupted. Hence, clearing the database.
EXT4-fs error (device sda2): htree_dirblock_to_tree: bad entry in directory #8554: directory entry across blocks - block=8120offset=528(53776), inode=11000, rec_len=4096, name_len=6
EXT4-fs error (device sda2): ext4_lookup: deleted inode referenced: 11909
rm: cannot unlink `/etc/fabos/Dcmd/WaveDatabase/base/16384/102983': Input/output error
EXT4-fs error (device sda2): ext4_lookup: deleted inode referenced: 10963
rm: cannot unlink `/etc/fabos/Dcmd/WaveDatabase/base/16384/100959': Input/output error
EXT4-fs error (device sda2): htree_dirblock_to_tree: bad entry in directory #8554: directory entry across blocks - block=8120offset=528(53776), inode=11000, rec_len=4096, name_len=6
EXT4-fs error (device sda2): htree_dirblock_to_tree: bad entry in directory #8554: directory entry across blocks - block=8120offset=528(53776), inode=11000, rec_len=4096, name_len=6
rm: cannot remove directory `/etc/fabos/Dcmd/WaveDatabase/base/16384': Directory not empty
rm: cannot remove directory `/etc/fabos/Dcmd/WaveDatabase/base': Directory not empty
rm: cannot remove directory `/etc/fabos/Dcmd/WaveDatabase': Directory not empty
.......EXT4-fs error (device sda2): htree_dirblock_to_tree: bad entry in directory #8554: directory entry across blocks - block=8120offset=528(53776), inode=11000, rec_len=4096, name_len=6
EXT4-fs error (device sda2): ext4_lookup: deleted inode referenced: 11909
EXT4-fs error (device sda2): ext4_lookup: deleted inode referenced: 10963
EXT4-fs error (device sda2): htree_dirblock_to_tree: bad entry in directory #8554: directory entry across blocks - block=8120offset=528(53776), inode=11000, rec_len=4096, name_len=6
EXT4-fs error (device sda2): htree_dirblock_to_tree: bad entry in directory #8554: directory entry across blocks - block=8120offset=528(53776), inode=11000, rec_len=4096, name_len=6
.EXT4-fs error (device sda2): htree_dirblock_to_tree: bad entry in directory #8554: directory entry across blocks - block=8120offset=528(53776), inode=11000, rec_len=4096, name_len=6
EXT4-fs error (device sda2): ext4_lookup: deleted inode referenced: 11909
EXT4-fs error (device sda2): ext4_lookup: deleted inode referenced: 10963
EXT4-fs error (device sda2): htree_dirblock_to_tree: bad entry in directory #8554: directory entry across blocks - block=8120offset=528(53776), inode=11000, rec_len=4096, name_len=6
EXT4-fs error (device sda2): htree_dirblock_to_tree: bad entry in directory #8554: directory entry across blocks - block=8120offset=528(53776), inode=11000, rec_len=4096, name_len=6
********************************************************************************************************
** Crashed in OM/Worker (WaveNs::DatabaseObjectManagerInstallWorker::installDatabaseStep(WaveNs::PrismLinearSequencerContext*))
********************************************************************************************************

WaveNs::DatabaseObjectManagerInstallWorker::installDatabaseStep(WaveNs::PrismLinearSequencerContext*)
WaveNs::PrismLinearSequencerContext::executeCurrentStep()
WaveNs::PrismLinearSequencerContext::executeNextStep(unsigned int const&)
WaveNs::DatabaseObjectManagerInstallWorker::installValidateStep(WaveNs::PrismLinearSequencerContext*)
WaveNs::PrismLinearSequencerContext::executeCurrentStep()
WaveNs::PrismLinearSequencerContext::start()
WaveNs::DatabaseObjectManagerInstallWorker::install(WaveNs::WaveAsynchronousContextForBootPhases*)
WaveNs::WaveObjectManager::installInstallWorkersStep(WaveNs::PrismLinearSequencerContext*)
WaveNs::PrismLinearSequencerContext::executeCurrentStep()
WaveNs::PrismLinearSequencerContext::start()
WaveNs::WaveObjectManager::installHandler(WaveNs::PrismInstallObjectManagerMessage*)
WaveNs::WaveObjectManager::PrismOperationMapContext::executeMessageHandler(WaveNs::PrismMessage*&)
WaveNs::WaveObjectManager::handlePrismMessage(WaveNs::PrismMessage*)
WaveNs::PrismThread::start()
WaveNs::PrismPosixThread::pthreadStartMethod(WaveNs::PrismPosixThread*)
/lib/libpthread.so.0 [0xe0f8e58]
clone

...........................................................................
 disk->start 7665663  cmd->start 0 nstart 7665663 disk name /vsmgr/vd@usb0/vda01
HV>
create_new_partition_table part_no 2 no_of_parts 2
HV> .......................................................................................................................................................................................................................................................................................................ERROR  : Fri Mar 22 13:14:51 2019 : srmd0 Fail to connect to WaveServer ipaddr=127.0.0.1 status=655364
.Fri Mar 22 13:14:51 JST 2019 :: Confd: Waiting for Dcmd to become ready...
...................................................................................................................................................................................................................................................................................................................................................................................................................................ERROR  : Fri Mar 22 13:21:51 2019 : srmd0 Fail to connect to WaveServer ipaddr=127.0.0.1 status=655364
......................................................2019/03/22-13:22:44, [HASM-1002], 26030, SW/0 | Active | FFDC, CRITICAL, VDX6740T-1G, Error happens on service instance chassis 0: command load failed or timed out (Critical).
2019/03/22-13:22:44, [RAS-1001], 26031, SW/0 | Active, INFO, VDX6740T-1G, First failure data capture (FFDC) event occurred.
....../bin/cat: /proc/fss/*/*: No such file or directory
cat: /var/log/fsslo.log: No such file or directory
.................................................................................................................................................................................................................................................................................................


Network OS (sw0)


sw0 console login: .......................................................................................ERROR  : Fri Mar 22 13:28:51 2019 : srmd0 Fail to connect to WaveServer ipaddr=127.0.0.1 status=655364
.............

 
Environment
  • VDX6740T-1G
  • NOS 5.0.1d
Cause
In extremely rare instances, during normal operation sudden file corruption can cause a VDX to reboot and fail to boot afterwards.
Resolution
Recovery is possible through netinstall. After netinstall, stability of the CF drive can be confirmed with badblocks.

For more information on netinstall, please see How to perform netinstall on VDX 6740 and 6940 via USB? or TFTP/NFS netinstall on 6740/6940/8770.

For more information on badblocks, please see How to check a VDX for CF Issue using the “badblocks” command.
Additional notes

Feedback

 

Was this article helpful?


   

Feedback

Please tell us how we can make this article more useful.

Characters Remaining: 255