PrerequisiteBefore you begin, assess if you can recover from fallback. If more than a single AMP in the same fallback cluster is down, you cannot use this procedure.
- SSH into one of the nodes.
- Run vprocmanager.
- Run the command status.
- Look for AMPs marked as FATAL** and check the column Cluster/Host No.If two AMPs with the same Cluster/Host No. are Fatal, then recovery using fallback is impossible. In the following example, both AMPs in cluster 0, 3, 6, 9 are Fatal and are not recoverable.
SYSTEM NAME: MPP160 17/01/12 14:40:21 DBS LOGICAL CONFIGURATION ------------------------- Rcv Jrnl/ Vproc Rel. Node Can Crash Vproc Config Config Cluster/ Host TVS Number Vproc# ID Move Count State Status Type Host No. Type Vproc ------ ------ ------ ----- ----- ------- -------- ------ -------- ----- ----- 0 1 1-01 Yes 0 FATAL** Online AMP 0 On 28671 1 2 1-01 Yes 0 FATAL** Online AMP 1 On 28671 2 3 1-01 Yes 0 FATAL** Online AMP 2 On 28671 3 4 1-01 Yes 0 FATAL** Online AMP 3 On 28671 4* 5 1-01 Yes 0 ONLINE Online AMP 4 On 28670 5 6 1-01 Yes 0 ONLINE Online AMP 5 On 28670 6 7 1-01 Yes 0 ONLINE Online AMP 6 On 28670 7 8 1-01 Yes 0 ONLINE Online AMP 7 On 28670 8 9 1-01 Yes 0 ONLINE Online AMP 8 On 28669 9 10 1-01 Yes 0 ONLINE Online AMP 9 On 28669 10 11 1-01 Yes 0 ONLINE Online AMP 10 On 28669 11 12 1-01 Yes 0 ONLINE Online AMP 11 On 28669 12 1 1-02 Yes 0 FATAL** Online AMP 0 On 28668 13 2 1-02 Yes 0 FATAL** Online AMP 3 On 28668 14 3 1-02 Yes 0 FATAL** Online AMP 6 On 28668 15 4 1-02 Yes 0 FATAL** Online AMP 9 On 28668
- If recovery is possible in your vSphere client, delete the failed pdisks that correspond to the fatal AMPs.
- Run tdc.deploy.ps1 with the storage step using the deployed system's Node.IT.Properties and Common.IT.PropertiesThis re-creates the removed pdisks.A copy of these two configs is located in /root/tvme on any of the nodes.
- When the command finishes, run tdc-rebuild as the root user.
- Enter your DBC account and password, then follow the prompts until the system is rebuilt.