Job Example 10: Loading Hadoop Files Using the HDFS API Interface - Parallel Transporter

Job Example 10: Loading Hadoop Files Using the HDFS API Interface

Job Objective:

The Teradata Parallel Transporter sample script loads five rows from a flat file located in Hadoop HDFS to a Teradata table.

Data Flow Diagrams

Figure 34 shows a flow diagram of the elements in Job Example 10.

Figure 34: Job Example PTS00029 -- Read HDFS and Load into Teradata

Sample Script

For the sample script that corresponds to this job, ee the following script in the sample/userguide directory:

PTS00029: Read HDFS flat file.

Rationale

This job uses:

DataConnector operator template as the producer because it can read files in the HDFS filesystem, referencing values defined in the job variable file without requiring an explicit operator definition.

Load operator template as the consumer because it is the consumer operator that offers the best performance for high speed writing of a large number of rows into an empty Teradata Database.