15.10 - Job Example 10: Loading Hadoop Files Using the HDFS API Interface - Parallel Transporter

Teradata Parallel Transporter User Guide

prodname
Parallel Transporter
vrm_release
15.10
category
User Guide
featnum
B035-2445-035K

Job Example 10: Loading Hadoop Files Using the HDFS API Interface

Job Objective:

The Teradata Parallel Transporter sample script loads five rows from a flat file located in Hadoop HDFS to a Teradata table.

Data Flow Diagrams

Figure 34 shows a flow diagram of the elements in Job Example 10.

Figure 34: Job Example PTS00029 -- Read HDFS and Load into Teradata

Sample Script

For the sample script that corresponds to this job, ee the following script in the sample/userguide directory:

PTS00029: Read HDFS flat file.

Rationale

This job uses:

  • DataConnector operator template as the producer because it can read files in the HDFS filesystem, referencing values defined in the job variable file without requiring an explicit operator definition.
  • Load operator template as the consumer because it is the consumer operator that offers the best performance for high speed writing of a large number of rows into an empty Teradata Database.