15.00 - Diagnostic Startup Options - Teradata Database

Teradata Database Support Utilities

prodname
Teradata Database
vrm_release
15.00
category
Configuration
featnum
B035-1180-015K

Diagnostic Startup Options

WARNING:

The Filer diagnostic options should be used only by trained Teradata personnel.

The Filer diagnostic options are used to repair a corrupted file system or WAL log. Users of the diagnostic options should have sufficient knowledge to repair Teradata file system internal structures, which includes a thorough understanding of the following subjects:

  • Teradata file system internal operation and structures
  • Basic Teradata file system startup sequence
  • Write Ahead Logging (WAL)

    WAL is a log-based file system recovery scheme in which modifications to permanent data are written to a log file, the WAL log. The log file contains change records (Redo records) which represent the updates. At key moments, such as transaction commit, the WAL log is forced to disk. In the case of a reset or crash, Redo records can be used to transform the old copy of a permanent data block on disk into the version that existed at the time of the reset.

    By maintaining the WAL log, the permanent data blocks that were modified no longer have to be written to disk as each block is modified. Only the Redo records in the WAL log must be written to disk. This allows a write cache of permanent data blocks to be maintained.

    WAL protects all permanent tables and all system tables, but is not used to protect any type of spool tables, volatile tables, or the Transient Journal (TJ), because TJ records are stored in the WAL log.

    The WAL log is maintained as a separate file system from the normal table area. Whole cylinders are allocated to the WAL log, and it has its own index structure.

    The WAL log data is a sequence of WAL log records and includes the following:

  • Redo records, used for updating disk blocks and insuring file system consistency during restarts.
  • TJ records used for transaction rollback.
  • The WAL log is subject to many of the same problems as the normal file system, such as data corruption from hardware, software, and operational problems. Filer provides commands to display and modify the WAL log and its index so that trained personnel can diagnose and repair such problems.

    Database and File System Startup

    File system startup is part of the normal sequence of events that are part of Teradata Database system startup, as detailed below:

    Teradata Database Startup

    1 File system startup

    2 Start the AMPs

    3 Start the PEs

    4 Voting for Transaction Recovery

    5 Transaction Rollback

    File System Startup

    A normal startup of the Teradata Database file system starts the WAL log prior to starting the normal file system. The key phases of Teradata file system startup include the following:

  • Build the WAL Master Index (WMI).
  • This is the index structure for the WAL log. The WAL log contains the Redo records and the Transient Journal (TJ) records.

  • Perform Redo analysis and replay.
  • This scans the WAL log for Redo records and analyzes which records should be applied, and replays the necessary modifications. Redo processing must be performed before the MI is built to bring all the Cylinder Index (CI) structures up-to-date.

  • Build the Master Index (MI).
  • Because the WAL protocol allows for the delayed writing of modified blocks to their home disk addresses, the blocks on disk are only completely valid after the Redo processing is complete. When the processing is finished, the CIs have been updated to reflect any cylinder migrates or other modifications that were performed. At this time, the building of the MI can proceed.

    The specific File System startup sequence on each AMP is as follows:

    1 Get the list of cylinders for this AMP from Teradata Virtual Storage (VS).

    2 Validate Segment Zero (SEG0), the File Information Block (FIB), and MI.

    3 Get the FSYS Trace segment.

    4 Process the DEPOT blocks.

    5 Process the WAL log:

    a Build the WMI.

    b Perform Redo analysis.

    c Perform Redo replay.

    6 Start the File System.

    a If the MI is in memory and valid:

    i Clean up cylinder index descriptors (CIDs) with special flags.

    i Clean up cylinder index descriptors (CIDs) with special flags.

    ii Rebuild the free sector list.

    iii Clean up pending migrate flags.

    b If the MI is not in memory or is invalid:

    i Build the MI.

    i Build the MI.

    ii Complete pending cylinder migrations.

    c Validate cylinders in FSYS structures against Teradata VS list.

    7 Start File System background tasks.

    When a system is corrupted, some or all of the above steps may be difficult to complete. The Filer diagnostic options allow trained personnel to start Filer even when errors are encountered during file system startup. The options also provide the ability to control which phases of the file system startup sequence are executed. This allows users to skip the startup phases which have errors and to use Filer to troubleshoot and repair the problems.

    Errors encountered during WAL log startup or during Redo processing will generally abort the startup. You can use the Filer diagnostic options to analyze the problems and possibly recover from the errors.

    Diagnostic Startup Syntax

    To use the Filer diagnostic startup options, include them on the command line when starting Filer from the Supervisor window of Database Window.

    The diagnostic options are described below. For an explanation of the non-diagnostic Filer startup options see “Standard Startup Syntax” on page 62.

     

    Syntax Element

    Description

    -u

    Does not build the WMI, does not perform any Redo processing, but builds the MI.

    If a problem exists such that the WAL log cannot be built successfully, first start Filer with the -x option. (If Filer does not present a command prompt, try restarting Filer using both the -x and -w options.) Repair the log, and restart Filer to perform Redo processing.

    If Filer does not present a command prompt when started with the options above, restart Filer using the -u option to skip building the WAL log and performing Redo operations. Examine and repair the problem WAL cylinder indexes (WCIs) and WAL data blocks (WDBs), and restart Filer again to perform Redo processing.

    If the WAL log is beyond repair, use the -u option again and run SCANDISK to determine the health of the file system before any decision is made on how to proceed next (such as deleting the complete WAL log).

    -w

    Executes the normal file system startup process, but ignores most errors. Redo processing continues even if errors are encountered during the Redo phase.

    If the -w option is used with another diagnostic option, the -r option need be specified only once. For example:

    start filer -w -y -r 2

    Use this option to clean up the WAL log and the file system as much as possible.

    This option skips certain file system startup errors, such as software errors related to the Redo process and its prerequisite conditions. Read errors and other hardware-related errors are not skipped.

    The Redo process attempts to process as much of the WAL log as possible, and then the rest of the normal file system startup logic is executed.

    It is highly recommended to run SCANDISK after the Redo process to determine the structural integrity of the file system. See “SCANDISK” on page 188.

    In cases where errors are encountered before the Redo phase, (for example, while building the WAL log), you can use the -w option together with another diagnostic option such as -z to skip both the errors and the Redo phase. This allows you to start Filer and possibly repair the WAL log so that subsequent file system startup and Redo processing attempts will succeed.

    A log file is created for each vproc where Filer -w is invoked. You can check these files for errors and warnings. See “Filer -w Error Log” on page 70.

    -x

    Builds the WAL Master Index (WMI), but does not perform Redo processing, and does not build the Master Index (MI).

    Use this option to examine and repair a corrupted WAL log. The option is useful when errors are encountered during Redo processing or when building the MI.

    The option provides the ability to examine and repair the WMI. It allows you to inspect the WAL log without trying to process the log or build the file system.

    It is highly recommended to run SCANDISK WAL to determine the structural integrity of the WAL index structure. See “SCANDISK” on page 188.

    Once repairs are completed, you can run Filer again to process the Redo records.

    -y

    Builds the WMI and performs Redo processing, but does not build the MI.

    Use this option when the file system index structure is corrupt, and building the MI causes Filer to abort. This option gives you the ability to examine and repair the index structure.

    Once repairs are completed, run Filer again to build the MI.

    -z

    Builds the WMI, does not perform Redo processing, but does build the MI.

    Use this option when errors are encountered during Redo processing, and the WAL log may be damaged beyond repair.

    You can examine the file system to determine whether it can be salvaged. You can determine whether it is possible to repair the WAL log sufficiently so that Redo processing using the -w option can complete, or to ignore the WAL log and just repair the file system index structure.

    It is highly recommended to run SCANDISK WAL to determine the structural integrity of the WAL log and the file system. See “SCANDISK” on page 188.

    -n

    Does not build the WMI, does not perform Redo processing, and does not build the MI.

    Use this option when serious problems exist in the file system which cause file system startup to fail in the early stages.

    This option executes as little file system startup logic as possible to bypass errors so that Filer can be started. You can examine the file system to determine whether it can be salvaged. Since key global file system structures are not rebuilt in this mode, some commands, including a full SCANDISK, will be disabled.

    -r vprocs

    Starts Filer in the -w, -u, -x, -y, -z, or -n diagnostic mode on the specified vprocs. The -r option must be used in conjunction with the other diagnostic options to limit the scope of these diagnostic modes. vprocs can be one of the following:

  • A single vproc number
  • A comma-separated list of vproc numbers
  • A range of vproc numbers in the form of start:end. For example, 2:5 represents vprocs 2, 3, 4, and 5.
  • A combination of vproc numbers and ranges of vproc numbers separated by commas
  • The syntax for vprocs is as follows:

    Filer will start using the specified diagnostic option on the vprocs specified by the vprocs argument. Filer will start normally on the other vprocs.

    Note: If Filer is started with the -v option to run Filer only on a single vproc, the number of the vproc where Filer is running must be included in the -r vprocs specification.

    -t

    Causes Filer invocation to create diagnostic files (dbgfil* and dbgftr*), which can be used to debug problems related to the Teradata Database File System.

    Diagnostic Option Usage Notes

    Running Filer with the diagnostic options is only allowed on vprocs that are in a fatal state or where the Teradata Database is not running. The -r option must be used with the diagnostic options to specify the AMPs on which Filer should run diagnostically. On other AMPs, Filer will run normally.

    The -u, -x, -y, -z, and -n options are mutually exclusive; however, the -w option can be used in combination with the -u, -x, -y, -z, or -n options.

    Note the distinction between the -v and the -r vprocs options of Filer:

  • -v runs Filer on a single AMP vproc. To specify the vproc on which Filer runs, the -v option of Filer is used together with the -v vproc option of the Database Window START command. If the START command -v vproc option is not specified, Filer runs on the control vproc of the node on which Filer was invoked.
  • -r vprocs runs Filer on all AMP vprocs in the system, but runs it in a diagnostic mode on the AMP vprocs specified by vprocs. The -r vprocs option must be used in conjunction with one or more of the other diagnostic Filer options. Filer runs its normal, non-diagnostic mode on the other AMPs in the system, which were not specified by ‑r vprocs.
  • When using any of the diagnostic options, SCANDISK should be run at the level that is available to the option. Initial results from SCANDISK will indicate whether the file system structures are coherent. If repairs are made based on the SCANDISK result, SCANDISK should be run again, either immediately after the repairs, or after quitting and restarting Filer with the appropriate options. This will insure that the repairs resulted in a structurally correct file system. See “SCANDISK” on page 188.

    Some Filer commands are not available with certain diagnostic modes because these commands depend on specific file system structures which may not be built when Filer is started with those diagnostic options.

    The following table shows the commands that are disabled for the various diagnostic options. Commands not listed in the table are valid in all modes of Filer.

     

     

    -u

    -w

    -x

    -y

    -z

    -n

    CID

     

     

    Disabled

    Disabled

     

    Disabled

    FIND

     

     

    Disabled

    Disabled

     

    Disabled

    MI

     

     

    Disabled

    Disabled

     

    Disabled

    MODIFY

     

     

     

     

     

    Disabled

    ROW

     

     

    Disabled

    Disabled

     

    Disabled

    SCANDISK (Full)

    Disabled

     

    Disabled

    Disabled

     

    Disabled

    SCANDISK WAL

    Disabled

     

     

     

     

    Disabled

    SCANDISK WMI

    Disabled

     

     

     

     

     

    TABLE

     

     

    Disabled

    Disabled

     

    Disabled

    WABC

    Disabled

     

     

     

     

     

    WAL

    Disabled

     

     

     

     

    Disabled

    WCID

    Disabled

     

     

     

     

    Disabled

    WFIND

     

     

     

     

     

    Disabled

    WFLUSH

    Disabled

     

     

     

     

    Disabled

    WMI

    Disabled

     

     

     

     

    Disabled

    WREC

    Disabled

     

     

     

     

    Disabled

    WSUMMARY

     

     

     

     

     

    Disabled

    The diagnostic options have no direct effect on the Transient Journal (TJ). However, if manual repairs are made to the WAL log, the TJ records might also be affected.

    When Filer is started with any of the diagnostic options, an informational message will be appended to the streams log(s) specifying which vprocs were affected.

    Filer -w Error Log

    When Filer is started with the -w option, a separate log file is created for each vproc where Filer -w is invoked. By default, the log is created in /var/opt/teradata/tdtemp. The location and name of the file is displayed on the Filer screen. The file logs any warnings and errors encountered during Filer startup on that vproc.

    After Filer is started, users should check the log files for errors or warnings, and take the appropriate action to ensure that the file system structures are consistent. The log file is useful for diagnosing WAL log problems, and identifying tables that may need repair.

    Example  

    The following example shows the Filer screen when Filer is invoked with the -w option. The output screen displays the location and names of the log files created.

    To start Filer -w diagnostic mode on vproc 1, and have Filer run in normal mode on other vprocs in the system, type the following into the Supervisor window of Database Window:

    start filer -w -r 1

    Filer shows the following output in the Filer Application window of Database Window, and subsequent Filer commands are entered into this window:

    Filer will be started with the -w option on the following VPROCs
       VPROC(s) 00001 (0001)

    Filer will be started normally on all other VPROCs

    WARNING: The Filer option that you've entered allows you to run Filer in a special diagnostic mode. Running in this mode can cause serious damage to your data if you do not understand both the internals of the file system and the basic startup sequence. Only trained field support personnel should attempt to use these options.

    Do you wish to continue?? (Y/N)
    y

     

    File System Debug Utility
     _______
        |                                |              |
        |    ___     __     ____         |    ____    __|__    ____
        |   /      |/  \    ____|    ____|    ____|     |      ____|
        |   ---    |       /    |   /    |   /    |     |     /    |
        |   \___   |       \____|   \____|   \____|     |__   \____|

        Release 14.00.00.00 Version 14.00.00.00
        Filer

    Waiting for Filer Slave tasks to start
    Command has been sent to Slave tasks.


    vproc 1 (0001) response

    Filer -w started on VPROC 1.
    Startup information and errors will be logged
    in the file /var/opt/teradata/tdtemp/filer.w1.120938.txt.

    Mon Apr 07, 2011 18:19:18

     4 of 4 amps have been selected

    Filer ==>

    Example  

    The following example starts Filer with the -w option on vprocs 2, 3 and 4, but starts Filer normally (without the -w option) on all other vprocs.

    start filer -w -r 2, 3, 4

    This command can also be invoked using the following syntax:

    start filer -w -r 2:4

    Example  

    The following example starts Filer with the -y option on vprocs 1, 4 through 7, and 9, but starts Filer normally on all other vprocs.

    start filer -y -r 1, 4:7, 9

    Example  

    The following example starts Filer with the -n option on vproc 3 only. Since the Filer -v option (the second -v) is used, the target vproc (vproc 3 as specified by the -v 3 option) must be listed in the arguments for the -r option.

    start -v 3 filer -v -n -r 3