17.10 - Initialization String - Access Module

Teradata® Tools and Utilities Access Module Reference

Product
Access Module
Release Number
17.10
Published
October 2021
Language
English (United States)
Last Update
2021-11-02
dita:mapPath
uur1608578381725.ditamap
dita:ditavalPath
obe1474387269547.ditaval

The following is the basic syntax of the Teradata Access Module for GCS initialization string for writing data to GCS and reading data from GCS. The initialization string consists of a series of keyword=value pairs, each pair separated by blanks. The value of a keyword can be an integer or string (the string does not need to be enclosed in quotes unless it has embedded spaces). For example:

AccessModuleInitStr = 'Bucket=roger Prefix=monday Object=data_load_1'


where:

Syntax Element Description
AccessID (Required) This parameter is effectively the GCS user id. The value of AccessID can be specified in the initialization string parameter AccessID, the environment variable GCS_ACCESS_KEY_ID, or the GCS credentials file. If specified by the initialization string parameter AccessID, the value of the environment variable GCS_ACCESS_KEY_ID and the contents of the GCS credentials file will not be considered. If set by the environment variable GCS_ACCESS_KEY_ID, the contents of the GCS credentials file will not be considered.

Consider your security policy when deciding if you want to store this value in your job variable file.

You can also store the AccessID in Teradata Wallet as long as your item name begins with the mandatory prefix “tptgcs_”.

AccessKey (Required) This parameter is effectively the GCS password. The value of AccessKey can be specified in initialization string parameter AccessKey, the environment variable GCS_SECRET_ACCESS_KEY, or the GCS credentials file. If specified by the initialization string parameter AccessKey, the value of the environment variable GCS_SECRET_ACCESS_KEY and the contents of the GCS credentials file will not be considered. If specified by the environment variable GCS_SECRET_ACCESS_KEY, the contents of the GCS credentials file will not be considered.

Consider your security policy when deciding if you want to store this value in your job variable file.

You can also store the AccessKey in Teradata Wallet as long as your item name begins with the mandatory prefix “tptgcs_”.

Bucket (Required) Specifies the GCS bucket to be used for read and write operations.

This parameter must be listed in the access module parameters initialization string.

BufferCount (Optional) Specifies the number of buffers to be used with the TCP connections specified by the ConnectionCount.

Twice the number of connections is the minimum recommended value. If not specified, the value of (2*ConnectionCount) is used as default.

BufferSize (Optional) Specifies the size of the buffers to be used for the TCP connections. The default is 8 MB (8388608 bytes). For convenience the following multipliers can be used:
  • k (1000)
  • K (1024)
  • m (1000 * 1000)
  • M (1024*1024)
For example, the default could be specified as 8M or 8192K.
ConfigDir (Optional) GCS provides for application programs to obtain some frequently needed parameters from a credentials file stored in a configuration directory. By convention, this directory is $HOME/.gcs, but may be overridden with this parameter.
ConnectionCount (Optional) Specifies the number of TCP connections to the GCS service.

If not specified, the default of 10 is used.

HttpsProxy (Optional) Specifies the IP address (or domain name) and port number of an HTTPS Proxy that does not require authentication in the format https://w.x.y.z:m where w.x.y.zis the IP address of the proxy server and m is the port number of the proxy service. Additional details on this parameter are discussed in HTTPS Proxy Support.
LogAPI (Optional) Creates a debug trace file with additional information for developer use. Because the trace file will contain the user's AccessID, the LogAPI parameter should not be used except at the express direction of Teradata.
MaxObjectSize (Optional) This parameter applies only for write operations to GCS.
The value of MaxObjectSize can be an integer or an integer followed by, without a space, one of the following multipliers:
  • k (1000)
  • K (1024)
  • m (1000*1000)
  • M (1024*1024)
For example, MaxObjectSize=100m will cause the Teradata Access Module for GCS to close the current object when it got to 100,000,000 bytes, and then open a new object. The Teradata Access Module for GCS will always write an integral number of TPT buffers, so objects may be closed somewhat before the specified size is reached. In addition, the Teradata Access Module for GCS must write at least one buffer to each object, so very small values of MaxObjectSize may result in objects in the 1-8 Megabyte range.

The naming convention for the objects changes when MaxObjectSize is used in the AccessModuleInitStr. Each object will be renamed to:

<object-name>-xxx

where xxx is a sequential 3-digit number, beginning with 001.

If the object name has a file extension, the 3-digit number will be appended to the base name, prior to the extension:

<object-name>-xxx.<extension>

Object (Required) The name of the object to be created in the Google Cloud Storage bucket. The pathname of the object comprises Object, appended to the value of Prefix.
Prefix (Optional) Is prepended to the Object string to create the pathname within the bucket. As a convenience in writing export or load scripts, this parameter could be used for the portion of the pathname that does not change when a series of tables are exported or loaded.
Within a GCS bucket, there is no need to explicitly create directories. The GCS CLI and GUI will segregate the listing of objects based on the presence of “/”, but this is a display convenience only. You do not need to create directories to use a “/” in a pathname, and the presence or location of one or more “/” in your pathname can come from either or both the Prefix or Object strings. In particular, the Prefix, if used, does not need to contain or end in a “/”.
Profile (Optional) The credentials file is broken into sections. Each section is labeled with a bracketed heading. The Profile variable selects a section in the credentials file. There must always be a section called [default]. This section will be used if no Profile is specified or if the value default is explicitly listed as default: Profile=default.