Job Example 10: Loading Hadoop Files Using the HDFS API Interface - Parallel Transporter

Teradata® Parallel Transporter User Guide - 17.20

Deployment
VantageCloud
VantageCore
Edition
Enterprise
IntelliFlex
Lake
VMware
Product
Parallel Transporter
Release Number
17.20
Published
June 2022
Language
English (United States)
Last Update
2023-08-25
dita:mapPath
uzp1645128359760.ditamap
dita:ditavalPath
tvt1507315030722.ditaval
dita:id
B035-2445
Product Category
Teradata Tools and Utilities

Job Objective

The Teradata Parallel Transporter sample script loads five rows from a flat file located in Hadoop HDFS to a database table.

Data Flow Diagram

The following figure shows a flow diagram of the elements in Job Example 10.

Job Example PTS00029 – Read HDFS and Load into the Database

Sample Script

For the sample script that corresponds to this job, see the following script in the sample/userguide directory:

PTS00029: Read HDFS flat file.

Rationale

This job uses:
  • DataConnector operator template as the producer because it can read files in the HDFS filesystem, referencing values defined in the job variable file without requiring an explicit operator definition.
  • Load operator template as the consumer because it is the consumer operator that offers the best performance for high speed writing of a large number of rows into an empty database table.