Use Multiple Data Mover Agents in Parallel | Teradata Data Mover - Using Multiple Data Mover Agents in Parallel - Teradata Data Mover

Teradata® Data Mover User Guide - 20.01

Deployment
VantageCloud
VantageCore
Edition
Enterprise
IntelliFlex
Lake
VMware
Product
Teradata Data Mover
Release Number
20.01
Published
November 2023
Language
English (United States)
Last Update
2023-12-05
dita:mapPath
hlv1700545853003.ditamap
dita:ditavalPath
mpm1591127278842.ditaval
dita:id
don1467241476387
Product Category
Analytical Ecosystem

Jobs that copy large amounts of data using Teradata PT API can distribute the copy tasks for a single database object across multiple agents. Performance is improved because Data Mover draws on the combined resources of data streams associated with each agent. When an agent uses multiple data streams to process a task in parallel, the object is copied faster.

The following restrictions apply when using multiple agents in parallel:
  • More than one agent must be installed in the Data Mover environment.
  • The job can only use Teradata PT API.
  • Jobs that use Teradata PT API only copy data in tables.
If Data Mover needs to perform additional SQL operations, this work is allocated to a single agent.
If max_agents_per_task is not specified or specified with no value, Data Mover dynamically sets the value for optimal performance.

The following parameters file for the create command assumes that the Data Mover environment has five online agents. Settings in this file are valid for jobs that use Teradata PT API.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<dmCreate xmlns="http://schemas.teradata.com/dataMover/v2009"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://schemas.teradata.com/unity/datamover.xsd">
				<job_name>floyd_dmdev_create</job_name>
				<source_tdpid>floyd</source_tdpid>
    <source_user>dmguest</source_user>
    <source_password>please</source_password>
    <target_tdpid>dmdev</target_tdpid>
    <target_user>dmguest</target_user>
    <target_password>please</target_password> 
    <data_streams>4</data_streams>
    <max_agents_per_task>4</max_agents_per_task>
    <source_sessions>4</source_sessions>
    <target_sessions>4</target_sessions>
    <log_level>0</log_level>
    <database selection="unselected">
  			<name>dmguest</name>
				<table selection="included">
				<name>orders_2010</name>
				</table>
    </database>
</dmCreate>