Loading Using a Map File - Aster Client

Teradata Aster® Client Guide

Product
Aster Client
Release Number
7.00
Published
May 2017
Language
English (United States)
Last Update
2018-04-13
dita:mapPath
hki1475000360386.ditamap
dita:ditavalPath
Generic_no_ie_no_tempfilter.ditaval
dita:id
B700-2005
lifecycle
previous
Product Category
Software
  1. Prepare your data files for import. Each file should be formatted as usual for the Aster Loader Tool. All the files you submit in a single running of the Aster Loader Tool must be in the same format.
  2. [Optional] You can optionally pass connection parameters in the map file. Any connection parameters specified on the command line supercede those in the map file. You can specify the following connection parameters:
    • dbname - Name of the database
    • username - Name of the Aster Database user
    • password - Password of the Aster Database user
    • loader - Hostname or IP address of the Aster Loader node
    • force-loader - Same as the -f command line option. Instructs the Aster Loader Tool to use the loader node specified with the loader parameter even if the IP address provided is not known to Aster Database. Note that if this option is specified, the Aster Loader Tool will only try that single IP address and return an error status if the connection fails for any reason.
    • timeout - Same as the -t command line option. Specifies the timeout value in seconds for Aster Database connection attempts. Default is 30 seconds.
  3. Prepare your map file. Note that a map file loads files serially. The map file is a text file containing a set of logical text blocks, each surrounded by curly braces. Each block represents a file or directory to be loaded. The format is like this:
    {
    "dbname" : "beehive",
    "username" : "beehive",
    "password" : "beehive_pwd",
    "loader" : "141.206.66.28",
    "force-loader" : true,
    "timeout" : 5,
    "loadconfig" :
    [
    {
    "table" : "schema1.targettable1",
    "file" : "data/insert1.txt",
    "errorlogging" : { "enabled" : true }
    },
          {
    "table" : "schema1.targettable1",
    "file" : "data/insert2.txt",
    "begin-script" : "input/mapfile/begin-script.sql",
    "end-script" : "input/mapfile/end-script.sql",
    "errorlogging" :
    {
    "enabled" : true,
    "discard-errors" : true
    "label" : "insert2_log",
    "schema" : "nc_system",
    "table" : "nc_errortable_part"
    }
       ]
    }

    In the above example, we assume the current directory (from which we invoke ncluster_loader) contains a subdirectory, data, which has two files, insert1.txt and insert2.txt, and we load these both into table targettable1 in schema1. Error logging is turned on. For the second table, additional error logging parameters are supplied to log to a system table, label each row and skip errors.

    Each block in the map file must contain the following required parameter flags and their values:

    • "table" Specifies the name of the target table. Typically you will schema-qualify the table name as shown in the example above. You may omit the schema, in which case the user’s schema search path determines the schema. If your table name is case sensitive, you must surround the table name with backslash-escaped double quote marks (\"), like this:
      "table" : "schema1.\"TargetTable\"",
    • "file" Specifies the name of the file or directory to be loaded. See filename in the Syntax section.
    • "begin-script" and "end-script" Specify scripts to run either before or after the loading of the file. Both are optional. Each map file entry can have a separate begin-script and end-script. For each map file entry, ncluster_loader will run the begin-script, load the data from the file, then run the end-script. Do not include commands that begin or end the transaction in the script files. Empty lines in the scripts will be ignored.

      Here is an example of a valid script file:

      DROP table IF EXISTS foo;
      CREATE table foo (id int, sometext varchar(40)) DISTRIBUTE BY HASH (id);
    • "errorlogging" Introduces a block that specifies how to handle malformed rows in the input file. Inside the errorlogging block (enclosed in curly braces) you pass the enabled parameter and, optionally, the parameters discard-errors, label, limit, schema, and table. These parameters correspond, respectively, to the command-line flags, --el-enabled, --el-discard-errors, --el-label, --el-limit, --schema, --el-table.

    Each block in the map file can contain the optional parameter flags listed in the middle column of the table in the Argument Flags section.

  4. Run the Aster Loader Tool, passing the --map-file or -m option and the name of your map file. Pass additional command line flags as needed, observing the rules set forth in the preceding paragraph.