Forty Gig interfaces which were configured as breakout no longer show 10G sub-interfaces on 8770, even though switch configuration has the breakout config in place.
The memory condition caused by DEFECT000564080made LC1, LC3 and LC4 go into fault state after a software crash onan8770 switch.
The user performedpower-off/power-onon thefaulty LCs.The LCs came up fine, but Forty Gig interfaces that were configured as breakout no longer showed 10G sub-interfaces under the output of show ip interface brief.
show ip interface brief: Interface IP-Address Vrf Status Protocol ================================================================================= FortyGigabitEthernet 87/1/10 unassigned default-vrf up down FortyGigabitEthernet 87/1/11 unassigned default-vrf up down FortyGigabitEthernet 87/1/13 unassigned default-vrf up down
Software Release: NOS 5.0.1d Fixed in Version: N/A
The root cause for failed breakout interfaces was due to incorrect entries in the config database
The events that triggered the incorrect database entries werenot found, as most of the traces and logs pointing to breakout configuration were wrapped up/rolled over
In this case, the incorrect programmed entries in the database existed for more than a year;but the userdid not encounter the problem until the LCs were rebooted, due to the defect mentioned above
In this case, re-configuring port-groups did not work,due to stale entries in the database.
Additional assistance was required from Brocade's Software Development Team to recover the stale entries
If such an issue is encountered, first try to re-configure the missing config. If that doesn't fixthe problem,contact Brocade Technical Support for further assistance and recovery
Brocade Engineering provided a recovery script to clear stale port-group entries from internal database and torecover from theCommitting to Underlying Data Store failederror
The script wascopied to the root folder of both ACTIVE and STDBY MM modulesand was run fromroot prompt
The two arguments to the script wererbridge IDand the slot ID:
root> ~/cleanup_port_group_config.sh 87 1 <----Here the script is run against RB87 for slot 1 Deleting all port group config from DB which matches this slot 87/1 DELETE FROM wavecurrent.wavelocalconfigmanagedobject where configname like 'PortGroupConfigManagedObject*87/1/%'
The script needs to be run for all the affected linecardsand on both the MM modules
The script also needs to be performedacross all the rbridges in the fabric,as the config database is synced across theentire VCS fabric
Once this db cleanup was complete, the port-groups werereconfigured
NOTE: port-group configuration requires LC to be in power-off state.