16.10 - About Exporting Data - Access Module

Teradata Tools and Utilities Access Module Reference

prodname
Access Module
vrm_release
16.10
created_date
July 2017
category
Programming Reference
featnum
B035-2425-077K

The following figure provides an overview of how data is exported from a Teradata Database to the Kafka Server through Teradata Parallel Transporter exporting operators and the Teradata Access Module for Kafka.

Although the figure shows the Data Consumer, Kafka Server, Teradata Parallel Transporter exporting operators, and Teradata Access Module for Kafka on separate systems, these entities can be on the same system.

Exporting Data

  1. The Teradata Parallel Transporter exporting operators retrieve data from the Teradata Database and transfer the data to the Teradata Parallel Transporter DataConnector operator.
  2. The Teradata Parallel Transporter DataConnector operator initiates a sequence of instructions to transfer the data to the Teradata Access Module for Kafka.

    This sequence of instructions includes the access module Initialize, File Open, File Write, File Close and Shutdown.

  3. The Teradata Access Module for Kafka uses APIs of librdkafka, a third-party open source shared library, to connect to the Kafka Server.
  4. The Teradata Access Module for Kafka composes database records into messages and sends the messages to the Kafka Server by using APIs of librdkafka, a third-party open source shared library.
  5. The Kafka Server puts the messages on a specified partition of a certain destination, called a Topic, for consumption.