z/OS - Parallel Transporter

Teradata® Parallel Transporter Application Programming Interface Programmer Guide - 17.20
Product
Parallel Transporter
Release Number
17.20
Published
June 2022
Language
English (United States)
Last Update
2022-10-11
dita:mapPath
fag1645201363032.ditamap
dita:ditavalPath
obe1474387269547.ditaval
dita:id
B035-2516
Product Category
Teradata Tools and Utilities
  1. After installation, the TPTAPI samples reside in the following datasets where <TTU> is the installation prefix:
    TPT Datasets (for the TDP variant)
    Database Name Description
    <TTU>.SAMPLIB TTU Sample Library (includes TPT and TPTAPI)
    <TTU>.TPT.H TPTAPI SDK and Sample .H files
    TPT Datasets (for the Gateway variant)
    Database Name Description
    <TTU>.GTW.SAMPLIB TTU Sample Library (includes TPT and TPTAPI)
    <TTU>.GTW.TPT.H. TPTAPI SDK and Sample .H files

    The application libraries are as follows:

    TPT Application Libraries (for the TDP variant)
    Dataset Name Description
    <TTU>.TPTLOAD TPT/TPTAPI Application Load Library
    <TTU>.APPLOAD TTU/CLI Application Load Library
    <TTU>.SDSNMACS TPTAPI Definition Side Deck Import Library
    TPT Application Libraries (for the Gateway variant)
    Dataset Name Description
    <TTU>.GTW.TPTLOAD TPT/TPTAPI Application Load Library
    <TTU>.GTW.APPLOAD TTU/CLI Application Load Library
    <TTU>.GTW.SDSNMACS TPTAPI Definition Side Deck Import Library

    The TPT Sample Library PDS Member Naming Standards are as follow (8 bytes):

    PT t aaaaa

    where:

    TPT Sample Library PDS Member Naming Standards
    This Variable... Defines the...
    PT TPT product identifier
    t Type of the item
    Possible values include:
    • $= JCL item
    • # = JCL PROC item
    • @ = TPT Job Variables
    • B = BTEQ Script
    • S = TPT Script
    • C = C or C++ Program Sample
    • T = Text Document
    • I = Program Input
    • H = Program Header File
    • X = Notify Exit
    aaaaa Unique item identifier
    All of the below member names exist for both the TDP and Gateway variants.
    The Gateway variant of TPT uses the Gateway variant of CLI, which must run under POSIX(ON) environment. Hence, all the sample JCLs that run the TPTAPI applications now contain the runtime option CEEOPTS to set POSIX(ON). Similarly, all the sample CPP files that contain the main() function now use 'pragma runopts(POSIX(ON))'. POSIX(ON) can be used for the TDP variant too, just that it is mandatory when using the Gateway variant. Any of the following methods can be used to set the POSIX(ON) runtime option:
    • Using CEEOPTS in JCL. Example:

      //CEEOPTS DD *

      POSIX(ON)

      /*

    • Using PARM statement in JCL. Example:

      //GENERIC EXEC PGM=GENERIC1,PARM='POSIX(ON)/DD:INPUT

    • Using '#pragma runopts(POSIX(ON))' in the TPTAPI application code that contains the main() function.

    The Gateway variant of TPTAPI library will fail to connect with an error if POSIX is OFF.

    Starting with the TPT 17.10 version, the Gateway variant of TPTAPI supports a new attribute called TD_CONNECTSTRING. On z/OS, the TDP variant of TPTAPI does not support the TD_CONNECTSTRING attribute and the attribute will be ignored.
  2. Become familiar with the Teradata API methods in the sample programs.

    • The SDK consists of the following components:

      SDK *.H
      Unix Name z/OS Name
      connection.h CONNECTI
      DMLGroup.h DMLGROUP
      schema.h SCHEMA
      Dynamic DDL SDK *.H
      Unix Name z/OS Name
      connectionX.h CONNECTX
      DMLGroupX.h DMLGROUPX
      schemaX.h SCHEMAX

    • The Generic Driver example consists of the following components:

      Generic Driver *.CPP
      Unix Name z/OS Name Description
      main.cpp PTCMAIN Main Entry Point Console for the Generic Driver
      OptionsManager.cpp PTCOPTMA Options Manager
      GenericDriver.cpp PTCGENDR Generic Driver Example
      Generic Driver *.H
      Unix Name z/OS Name
      connection.h CONNECTI
      DMLGroup.h DMLGROUP
      schema.h SCHEMA
      GenericDriver.h PTCGENDR
      OptionsManager.h PTCOPTMA
      Generic Driver JCL
      Unix Name z/OS Name Description
      n/a PT$CMVS Compile and Bind the Generic Driver
      n/a PT$MVSGO Generic Drive Execution
      Generic Driver Configuration File 
      Unix Name z/OS Name Description
      n/a PTIGENDR Generic Drive Input
      setupTables.bteq PTBGDSET BTEQ script to set up generic driver test
      cleanupTables.bteq PTBGDREM BTEQ script to delete generic driver tables

    • The Dynamic DLL Generic Driver Example consists of the following components:

      Dynamic DLL Generic Driver *.CPP
      Unix Name z/OS Name Description
      main.cpp PTCMAIN Main Entry Point Console for the Generic Driver
      OptionsManager.cpp PTCOPTMA Options Manager
      GenericDriver.cpp PTCGENDD Generic Driver Example
      dynlib.cpp PTCDLLIB Dynamically Loads DLL
      Dynamic DLL Generic Driver *.H
      Unix Name z/OS Name
      connectionX.h CONNECTX
      DMLGroupX.h DMLGRPX
      schemaX.h SCHEMAX
      GenericDriver.h PTCGENDR
      OptionsManager.h PTCOPTMA
      dynlib.h PTCDLLIB
      Dynamic DLL Generic Driver JCL
      Unix Name z/OS Name Description
      n/a PT$CMVSD Compile and Bind the Generic Driver
      n/a PT$MVSDG Generic Drive Execution

    • The Multi Node Main consists of the following components:

      Multi Node Main *.CPP
      Unix Name z/OS Name Description
      fileio.cpp PTCFILIO File Input and Output for the multi-node Main and Worker
      master.cpp PTCMASTR Main for the multi-node example
      multitest.cpp PTCMULTI Multi-node routines
      OptionsManager.cpp PTCOPTMA Options Manager
      synchronize.cpp PTCSYNCR Multi-node synchronization routines
      Multi Node Main *.H
      Unix Name z/OS Name
      connection.h CONNECTI
      DMLGroup.h DMLGROUP
      schema.h SCHEMA
      fileio.h PTCFILIO
      synchronize.h PTCSYNCR
      multitest.h PTCMULTI
      OptionsManager.h PTCOPTMA
      Multi Node Main JCL
      Unix Name z/OS Name Description
      n/a PT$CMNMS Compile and Bind the Multi-Node Main
      n/a PT$MNMSG Execution JCL for the Multi-Node Main
      Multi Node Main Configuration Files 
      Unix Name z/OS Name Description
      n/a PTIMNMST Input for the Multi-Node Main
      setupTables.bteq PTBMNSET BTEQ script to set up multi-node test
      cleanupTables.bteq PTBMNREM BTEQ script to delete multi-node tables

    • The Dynamic DLL Multi Node Main consists of the following components:

      Dynamic DLL Multi Node Main *.CPP
      Unix Name z/OS Name Description
      fileio.cpp PTCFILIO File Input and Output for the multi-node Main and Worker
      master.cpp PTCMASTD Main for the multi-node example
      multitest.cpp PTCMULTD Multi-node routines
      OptionsManager.cpp PTCOPTMA Options Manager
      synchronize.cpp PTCSYNCD Multi-node synchronization routines
      dynlib.cpp PTCDLLIB Dynamically Loads the DLL
      Dynamic DLL Multi Node Main *.H
      Unix Name z/OS Name
      connectionX.h CONNECTX
      DMLGroupX.h DMLGRPX
      schemaX.h SCHEMAX
      fileio.h PTCFILIO
      synchronize.h PTCSYNCD
      multitest.h PTCMULTD
      OptionsManager.h PTCOPTMA
      dynlib.h PTCDLLIB
      Dynamic DLL Multi Node Main JCL 
      Unix Name z/OS Name Description
      n/a PT$CMNMD Compile and Bind the Multi-Node Main
      n/a PT$MNMDG Execution JCL for the Multi-Node Main

    • The Multi Node Worker consists of the following components:

      Multi Node Worker *.CPP
      Unix Name z/OS Name Description
      fileio.cpp PTCFILIO File Input and Output for the multi-node Main and Worker
      slave.cpp PTCSLAVE Worker for the multi-node example
      multitest.cpp PTCMULTI Multi-node routines
      OptionsManager.cpp PTCOPTMA Options Manager
      synchronize.cpp PTCSYNCR Multi-node synchronization routines
      Multi Node Worker *.H
      Unix Name z/OS Name
      connection.h CONNECTI
      schema.h SCHEMA
      DMLGroup.h DMLGROUP
      fileio.h PTCFILIO
      synchronize.h PTCSYNCR
      multitest.h PTCMULTI
      OptionsManager.h PTCOPTMA
      Multi Node Worker JCL
      Unix Name z/OS Name Description
      n/a PT$CMNS Compile and Bind the Multi-Node Worker
      n/a PT$MNSLG Execution JCL for the Multi-Node Worker
      Multi Node Worker Configuration Files
      Unix Name z/OS Name Description
      n/a PTIMNS Input for the Multi-Node Worker

    • The Dynamic DLL Multi Node Worker consists of the following components:

      Dynamic DLL Multi Node Worker *.CPP
      Unix Name z/OS Name Description
      fileio.cpp PTCFILIO File Input and Output for the multi-node Main and Worker
      slave.cpp PTCSLAVD Worker for the multi-node example
      multitest.cpp PTCMULTD Multi-node routines
      OptionsManager.cpp PTCOPTMA Options Manager
      synchronize.cpp PTCSYNCD Multi-node synchronization routines
      dynlib.cpp PTCDLLIB Dynamically Loads the DLL
      Dynamic DLL Multi Node Worker *.H
      Unix Name z/OS Name
      connectionX.h CONNECTX
      schemaX.h SCHEMAX
      DMLGroupX.h DMLGRPX
      fileio.h PTCFILIO
      synchronize.h PTCSYNCD
      multitest.h PTCMULTD
      OptionsManager.h PTCOPTMA
      dynlib.h PTCDLLIB
      Dynamic DLL Multi Node Worker JCL
      Unix Name z/OS Name Description
      n/a PT$CMNSD Compile and Bind the Multi-Node Worker
      n/a PT$MNSDG Execution JCL for the Multi-Node Worker

    • The Parallel Thread Example consists of the following components:

      Parallel Thread *.CPP
      Unix Name z/OS Name Description
      OptionsManager.cpp PTCOPTMA Options Manager
      PerfTestMessage.cpp PTCPERFT Performance Test Messages for the Thread Example
      TelapiThreadTest.cpp PTCTELTH Thread Test Main Console Application
      TestBase.cpp PTCBASET Thread Test: TestBase
      TestExport.cpp PTCEXPRT Thread Test: TestExport
      TestLoad.cpp PTCLOADT Thread Test: TestLoad
      TestSim.cpp PTCSIMT Thread Test: TestSim
      TestStream.cpp PTCSTRMT Thread Test: Test Stream
      TestUpdate.cpp PTCUPDAT Thread Test: Test Update
      ThreadTestParameters.cpp PTCTHDPA Thread Test Parameters
      ThreadTest.cpp PTCTHDTE Thread Test
      Synchronizer.cpp PTCTSYNC Thread Synchronizer
      Parallel Thread *.H 
      Unix Name z/OS Name
      common.h COMMON
      connection.h CONNECTI
      schema.h SCHEMA
      DMLGroup.h DMLGROUP
      OptionsManager.h PTCOPTMA
      PerfTestMessage.h PTCPERFT
      TestBase.h PTCBASET
      TestExport.h PTCEXPRT
      TestLoad.h PTCLOADT
      TestSim.h PTCSIMT
      TestStream.h PTCSTRMT
      TestUpdate.h PTCUPDAT
      ThreadTestParameters.h PTCTHDPA
      ThreadTest.h PTCTHDTE
      Synchronizer.h PTCTSYNC
      Parallel Thread JCL
      Unix Name z/OS Name Description
      n/a PT$CTHD Compile and Bind the Parallel Thread Example
      n/a PT$THDGO Execution JCL for the Parallel Thread Example
      Parallel Thread Configuration Files
      n/a PTIPAR Input for the Parallel Thread Example
      setupTables.bteq PTBTHSET BTEQ script to set up the Parallel Thread Example
      cleanupTables.bteq PTBTHREM BTEQ script to delete Parallel Thread Tables

    • The Dynamic DLL Parallel Thread example consists of the following components:

      Dynamic DLL Parallel Thread *.CPP 
      Unix Name z/OS Name Description
      OptionsManager.cpp PTCOPTMA Options Manager
      PerfTestMessage.cpp PTCPERTD Performance Test Messages for the Thread Example
      TelapiThreadTest.cpp PTCTELTD Thread Test Main Console Application
      TestBase.cpp PTCBASTD Thread Test: TestBase
      TestExport.cpp PTCEXPTD Thread Test: TestExport
      TestLoad.cpp PTCLODTD Thread Test: TestLoad
      TestSim.cpp PTCSIMTD Thread Test: TestSim
      TestStream.cpp PTCSTRTD Thread Test: Test Stream
      TestUpdate.cpp PTCUPDTD Thread Test: Test Update
      ThreadTestParameters.cpp PTCTHTPD Thread Test Parameters
      ThreadTest.cpp PTCTHDTD Thread Test
      Synchronizer.cpp PTCTSYND Thread Synchronizer
      dynlib.cpp PTCDLLIB Dynamically Loads the DLL
      Dynamic DLL Parallel Thread *.H
      Unix Name z/OS Name
      common.h COMMOND
      connectionX.h CONNECTX
      schemaX.h SCHEMAX
      DMLGroupX.h DMLGRPX
      OptionsManager.h PTCOPTMA
      PerfTestMessage.h PTCPERTD
      TestBase.h PTCBASTD
      TestExport.h PTCEXPTD
      TestLoad.h PTCLODTD
      TestSim.h PTCSIMTD
      TestStream.h PTCSTRTD
      TestUpdate.h PTCUPDTD
      ThreadTestParameters.h PTCTHTPD
      ThreadTest.h PTCTHDTD
      Synchronizer.h PTCTSYND
      dynlib.h PTCDLLIB
      Dynamic DLL Parallel Thread JCL 
      Unix Name z/OS Name Description
      n/a PT$CTHDD Compile and Bind the Parallel Thread Example
      n/a PT$THDDG Execution JCL for the Parallel Thread Example
  3. Add a Job Card and update the JCL variables to match the Installation Development Environment for the following JCL members:

    • PT$CMNMS: Compile the Multi Node Main Example

    • PT$CMNS: Compile the Multi Node Worker Example

    • PT$CMVS: Compile the Generic Driver Example

    • PT$CTHD: Compile the Parallel Thread Example

      Dynamic DLL

    • PT$CMNMD: Compile the Multi Node Main Example

    • PT$CMNSD: Compile the Multi Node Worker Example

    • PT$CMVSD: Compile the Generic Driver Example

    • PT$CTHDD: Compile the Parallel Thread Example

    The JCL uses the following standard JCL Variables (these should be updated to reflect the customer's requirements).

    Standard JCL Variables
    Name Default Value Description
    TTUPREF none (must be provided by customer) The installation dataset prefix
    SAMPLIB &TTUPREF..SAMPLIB The TTU sample library
    HFILES &TTUPREF..TPT.H The TPTAPI .H files
    DEVPREF none (must be provided by customer) The prefix for the developer’s private datasets
    OBJLIB &DEVPREF..OBJLIB The OBJECT library for the Compile JOBS
    SAMPLOAD &DEVPREF..LOADLIB The LOAD library for the Compile JOBS
    TPTLOAD &TTUPREF..TPTLOAD The TPT application Load Library
    APPLOAD &TTUPREF..APPLOAD The TTU/CLI application Load Library
    TPTDATA &TTUPREF..TPT.H The TPTAPI data is located in the .H Library
  4. Execute the compile jobs.
  5. Change the database machine name (TDP), database user name, and database user password in the following files:

    • PTBGDSET: Set up the Generic Driver Example

    • PTBGDREM: Tear down the Generic Driver Example

    • PTBMNSET: Set up the Multi Node Example

    • PTBMNREM: Tear down the Multi Node Example

    • PTBTHSET: Set up the Parallel Thread Example

    • PTBTHREM: Tear down the Parallel Thread Example

  6. Change the database machine name (TDP), database user name, and database user password in the following files:

    • PTIGENDR: The Generic Driver configuration parameters

    • PTIMNMST: The Multi Node Main configuration parameters

    • PTIMNS: The Multi Node Worker configuration parameters

    • PTIPAR: The Parallel Thread configuration parameters

  7. Add JOB cards and update the JCL variables in the test jobs:

    • PT$MNMSG: The Multi Node Main example

    • PT$MNSLG: The Multi Node Worker example

    • PT$MVSGO: The Generic Driver example

    • PT$THDGO: The Parallel Thread example

      Dynamic DLL:

    • PT$MNMDG: The Multi Node Main example

    • PT$MNSDG: The Multi Node Worker example

    • PT$MVSDG: The Generic Driver example

    • PT$THDDG: The Parallel Thread example

  8. Execute the test jobs.

    The JOB outputs should look something like the following:

    1) Multi Node Main Output:

Waiting for 1 workers
Accepted client 0
Received TD_SYNC_Barrier
Received TD_SYNC_TELINFO
Sending TELINFO of size 2288
TELINFO sync finished
Received TD_SYNC_Barrier
Received TD_SYNC_Barrier
Switching to barrier 1
Connection Initiate finished
Sent 50 rows
Received TD_SYNC_Barrier
Received TD_SYNC_Barrier
Switching to barrier 2
Connection EndAcquisition complete
Received TD_SYNC_Barrier
Received TD_SYNC_Barrier
Switching to barrier 0
Connection ApplyRows complete
Received TD_SYNC_Barrier
Switching to barrier 1
Connection terminated
Test completed
Press any key to continue

2)  Multi Node Worker Output

Connecting to 127.0.0.1
Connection accepted
Received TD_SYNC_Barrier
Received TD_SYNC_TELINFO
Receiving TELINFO of size 2288
TELINFO sync finished
Received TD_SYNC_Barrier
Received TD_SYNC_Barrier
Switching to barrier 1
Connection Initiate finished
Sent 50 rows
Received TD_SYNC_Barrier
Received TD_SYNC_Barrier
Switching to barrier 2
Connection EndAcquisition complete
Received TD_SYNC_Barrier
Received TD_SYNC_Barrier
Switching to barrier 0
Connection ApplyRows complete
Received TD_SYNC_Barrier
Switching to barrier 1
Connection terminated
Test completed
Press any key to continue

3) Generic Driver Output

*** Load Driver Example ***
Driver Initiated with status 3
max rec length is 251
End of file reached
Sent 10 rows
Acquisition completed with status 3
Rows Applied with status 3
Load completed successfully
Driver Terminated with status 3
Deleting objects
*** Load Complete ***

4) Parallel Thread Output

Multi-Threaded Load Test
 [Manager] Creating Thread 0/4
 [Manager] Creating Thread 1/4
 [Manager] Creating Thread 2/4
 obj location: fec2298
 obj location: fec2138
 [Manager] Creating Thread 3/4
 obj location: fec2558
 obj location: fec23f8
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [0]
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [0]
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [0]
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [0]
 [Manager] Barrier Reached
   [Thread 4] TELINFO Message
   [Thread 4] Waiting for manager [1]
   [Thread 3] TELINFO Message
   [Thread 3] Waiting for manager [1]
   [Thread 2] TELINFO Message
   [Thread 2] Waiting for manager [1]
   [Thread 1] TELINFO Message
   [Thread 1] Waiting for manager [1]
 [Manager] TELINFO Barrier Reached
 [Manager] Synchronizing TELINFO
 [Manager] TELINFO Synchronized
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [0]
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [0]
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [0]
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [0]
 [Manager] Barrier Reached
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [1]
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [1]
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [1]
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [1]
 [Manager] Barrier Reached
   [Thread 4] End of Method
   [Thread 4] Waiting for manager [0]
   [Thread 2] End of Method
   [Thread 2] Waiting for manager [0]
   [Thread 3] End of Method
   [Thread 3] Waiting for manager [0]
   [Thread 1] End of Method
   [Thread 1] Waiting for manager [0]
 [Manager] Barrier Reached
   [Thread 4] Driver Initiated
   [Thread 4] Sent row 0: 41634
   [Thread 4] Sent row 1: 1286818
   [Thread 4] Sent row 2: 3908258
   [Thread 4] Sent row 3: 6529698
   [Thread 4] Sent row 4: 2400930
   [Thread 4] Sent row 5: 6398626
   [Thread 4] Sent row 6: 1024674
   [Thread 4] Sent row 7: 4891298
   [Thread 4] Sent row 8: 3515042
   [Thread 4] Sent row 9: 2007714
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [1]
   [Thread 2] Driver Initiated
   [Thread 2] Sent row 0: 172706
   [Thread 2] Sent row 1: 1155746
   [Thread 2] Sent row 2: 2269858
   [Thread 2] Sent row 3: 893602
   [Thread 2] Sent row 4: 2138786
   [Thread 2] Sent row 5: 762530
   [Thread 2] Sent row 6: 3383970
   [Thread 2] Sent row 7: 631458
   [Thread 2] Sent row 8: 3252898
   [Thread 2] Sent row 9: 1876642
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [1]
   [Thread 3] Driver Initiated
   [Thread 3] Sent row 0: 2663074
   [Thread 3] Sent row 1: 5284514
   [Thread 3] Sent row 2: 3777186
   [Thread 3] Sent row 3: 3646114
   [Thread 3] Sent row 4: 6267554
   [Thread 3] Sent row 5: 4760226
   [Thread 3] Sent row 6: 500386
   [Thread 3] Sent row 7: 1483426
   [Thread 3] Sent row 8: 4104866
   [Thread 3] Sent row 9: 2597538
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [1]
   [Thread 1] Driver Initiated
   [Thread 1] Sent row 0: 4039330
   [Thread 1] Sent row 1: 2532002
   [Thread 1] Sent row 2: 5153442
   [Thread 1] Sent row 3: 5022370
   [Thread 1] Sent row 4: 6136482
   [Thread 1] Sent row 5: 1745570
   [Thread 1] Sent row 6: 369314
   [Thread 1] Sent row 7: 1614498
   [Thread 1] Sent row 8: 238242
   [Thread 1] Sent row 9: 2859682
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [1]
 [Manager] Barrier Reached
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [0]
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [0]
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [0]
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [0]
 [Manager] Barrier Reached
   [Thread 4] End of Method
   [Thread 4] Waiting for manager [1]
   [Thread 2] End of Method
   [Thread 2] Waiting for manager [1]
   [Thread 3] End of Method
   [Thread 3] Waiting for manager [1]
   [Thread 1] End of Method
   [Thread 1] Waiting for manager [1]
 [Manager] Barrier Reached
   [Thread 4] Checkpoint Taken
   [Thread 4] Checkpoint data: NULL, length: 0
   [Thread 4] TD_Evt_RowCounts reports 10 rows received
   [Thread 4] TD_Evt_RowCounts reports 10 rows sent
   [Thread 4] TD_Evt_RowCounts reports 10 rows applied
   [Thread 4] Sent row 10: 6005410
   [Thread 4] Sent row 11: 4629154
   [Thread 4] Sent row 12: 3121826
   [Thread 4] Sent row 13: 2990754
   [Thread 4] Sent row 14: 5612194
   [Thread 4] Sent row 15: 959138
   [Thread 4] Sent row 16: 2073250
   [Thread 4] Sent row 17: 1942178
   [Thread 4] Sent row 18: 4563618
   [Thread 4] Sent row 19: 434850
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [0]
   [Thread 2] Checkpoint Taken
   [Thread 2] Checkpoint data: NULL, length: 0
   [Thread 2] TD_Evt_RowCounts reports 10 rows received
   [Thread 2] TD_Evt_RowCounts reports 10 rows sent
   [Thread 2] TD_Evt_RowCounts reports 10 rows applied
   [Thread 2] Sent row 10: 5874338
   [Thread 2] Sent row 11: 4498082
   [Thread 2] Sent row 12: 5743266
   [Thread 2] Sent row 13: 4367010
   [Thread 2] Sent row 14: 4235938
   [Thread 2] Sent row 15: 5481122
   [Thread 2] Sent row 16: 5350050
   [Thread 2] Sent row 17: 1090210
   [Thread 2] Sent row 18: 2204322
   [Thread 2] Sent row 19: 696994
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [0]
   [Thread 3] Checkpoint Taken
   [Thread 3] Checkpoint data: NULL, length: 0
   [Thread 3] TD_Evt_RowCounts reports 10 rows received
   [Thread 3] TD_Evt_RowCounts reports 10 rows sent
   [Thread 3] TD_Evt_RowCounts reports 10 rows applied
   [Thread 3] Sent row 10: 2466466
   [Thread 3] Sent row 11: 5087906
   [Thread 3] Sent row 12: 2335394
   [Thread 3] Sent row 13: 3580578
   [Thread 3] Sent row 14: 6202018
   [Thread 3] Sent row 15: 828066
   [Thread 3] Sent row 16: 3449506
   [Thread 3] Sent row 17: 6070946
   [Thread 3] Sent row 18: 3318434
   [Thread 3] Sent row 19: 1811106
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [0]
   [Thread 1] Checkpoint Taken
   [Thread 1] Checkpoint data: NULL, length: 0
   [Thread 1] TD_Evt_RowCounts reports 10 rows received
   [Thread 1] TD_Evt_RowCounts reports 10 rows sent
   [Thread 1] TD_Evt_RowCounts reports 10 rows applied
   [Thread 1] Sent row 10: 107170
   [Thread 1] Sent row 11: 2728610
   [Thread 1] Sent row 12: 1352354
   [Thread 1] Sent row 13: 3973794
   [Thread 1] Sent row 14: 1221282
   [Thread 1] Sent row 15: 5218978
   [Thread 1] Sent row 16: 3842722
   [Thread 1] Sent row 17: 6464162
   [Thread 1] Sent row 18: 3711650
   [Thread 1] Sent row 19: 6333090
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [0]
 [Manager] Barrier Reached
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [1]
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [1]
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [1]
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [1]
 [Manager] Barrier Reached
   [Thread 4] End of Method
   [Thread 4] Waiting for manager [0]
   [Thread 2] End of Method
   [Thread 2] Waiting for manager [0]
   [Thread 3] End of Method
   [Thread 3] Waiting for manager [0]
   [Thread 1] End of Method
   [Thread 1] Waiting for manager [0]
 [Manager] Barrier Reached
   [Thread 4] Checkpoint Taken
   [Thread 4] Checkpoint data: NULL, length: 0
   [Thread 4] TD_Evt_RowCounts reports 20 rows received
   [Thread 4] TD_Evt_RowCounts reports 20 rows sent
   [Thread 4] TD_Evt_RowCounts reports 20 rows applied
   [Thread 4] Sent row 20: 3056290
   [Thread 4] Sent row 21: 5677730
   [Thread 4] Sent row 22: 1548962
   [Thread 4] Sent row 23: 5546658
   [Thread 4] Sent row 24: 1417890
   [Thread 4] Sent 25 Rows
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [1]
   [Thread 2] Checkpoint Taken
   [Thread 2] Checkpoint data: NULL, length: 0
   [Thread 2] TD_Evt_RowCounts reports 20 rows received
   [Thread 2] TD_Evt_RowCounts reports 20 rows sent
   [Thread 2] TD_Evt_RowCounts reports 20 rows applied
   [Thread 2] Sent row 20: 4694690
   [Thread 2] Sent row 21: 3187362
   [Thread 2] Sent row 22: 5808802
   [Thread 2] Sent row 23: 4301474
   [Thread 2] Sent row 24: 4170402
   [Thread 2] Sent 25 Rows
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [1]
   [Thread 3] Checkpoint Taken
   [Thread 3] Checkpoint data: NULL, length: 0
   [Thread 3] TD_Evt_RowCounts reports 20 rows received
   [Thread 3] TD_Evt_RowCounts reports 20 rows sent
   [Thread 3] TD_Evt_RowCounts reports 20 rows applied
   [Thread 3] Sent row 20: 4432546
   [Thread 3] Sent row 21: 303778
   [Thread 3] Sent row 22: 2925218
   [Thread 3] Sent row 23: 2794146
   [Thread 3] Sent row 24: 5415586
   [Thread 3] Sent 25 Rows
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [1]
   [Thread 1] Checkpoint Taken
   [Thread 1] Checkpoint data: NULL, length: 0
   [Thread 1] TD_Evt_RowCounts reports 20 rows received
   [Thread 1] TD_Evt_RowCounts reports 20 rows sent
   [Thread 1] TD_Evt_RowCounts reports 20 rows applied
   [Thread 1] Sent row 20: 4956834
   [Thread 1] Sent row 21: 4825762
   [Thread 1] Sent row 22: 5939874
   [Thread 1] Sent row 23: 565922
   [Thread 1] Sent row 24: 1680034
   [Thread 1] Sent 25 Rows
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [1]
 [Manager] Barrier Reached
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [0]
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [0]
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [0]
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [0]
 [Manager] Barrier Reached
   [Thread 2] End of Method
   [Thread 2] Waiting for manager [1]
   [Thread 4] End of Method
   [Thread 4] Waiting for manager [1]
   [Thread 3] End of Method
   [Thread 3] Waiting for manager [1]
   [Thread 1] End of Method
   [Thread 1] Waiting for manager [1]
 [Manager] Barrier Reached
   [Thread 2] Acquisition complete
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [0]
   [Thread 4] Acquisition complete
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [0]
   [Thread 3] Acquisition complete
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [0]
   [Thread 1] Acquisition complete
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [0]
 [Manager] Barrier Reached
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [1]
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [1]
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [1]
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [1]
 [Manager] Barrier Reached
   [Thread 2] End of Method
   [Thread 2] Waiting for manager [0]
   [Thread 4] End of Method
   [Thread 4] Waiting for manager [0]
   [Thread 3] End of Method
   [Thread 3] Waiting for manager [0]
   [Thread 1] End of Method
   [Thread 1] Waiting for manager [0]
 [Manager] Barrier Reached
   [Thread 2] Application complete
   [Thread 2] Calling Terminate
   [Thread 2] Returned from Terminate
   [Thread 2] Barrier Message
   [Thread 2] Waiting for manager [1]
   [Thread 4] Application complete
   [Thread 4] Calling Terminate
   [Thread 4] Returned from Terminate
   [Thread 4] Barrier Message
   [Thread 4] Waiting for manager [1]
   [Thread 3] Application complete
   [Thread 3] Calling Terminate
   [Thread 3] Returned from Terminate
   [Thread 3] Barrier Message
   [Thread 3] Waiting for manager [1]
   [Thread 1] Application complete
   [Thread 1] Calling Terminate
   [Thread 1] Returned from Terminate
   [Thread 1] Barrier Message
   [Thread 1] Waiting for manager [1]
 [Manager] Barrier Reached
   [Thread 2] Calling Terminate
   [Thread 2] Returned from Terminate
   [Thread 2] End of Method
   [Thread 2] Waiting for manager [0]
   [Thread 4] Calling Terminate
   [Thread 4] Returned from Terminate
   [Thread 4] End of Method
   [Thread 4] Waiting for manager [0]
   [Thread 3] Calling Terminate
   [Thread 3] Returned from Terminate
   [Thread 3] End of Method
   [Thread 3] Waiting for manager [0]
   [Thread 1] Calling Terminate
   [Thread 1] Returned from Terminate
   [Thread 1] End of Method
   [Thread 1] Waiting for manager [0]
 [Manager] Barrier Reached
   [Thread 2] Driver Terminated
   [Thread 2] Total Number of Rows Sent: 25
   [Thread 2] Number of Checkpoints Taken: 2
 [Manager] Instance 2 Terminated
   [Thread 2] Unread Msg:Thread Exiting
   [Thread 2] Unread Msg:Load Successful
   [Thread 2] Unread Msg:Load Thread Exited
 [Manager] 3 threads remaining
   [Thread 4] Driver Terminated
   [Thread 4] Total Number of Rows Sent: 25
   [Thread 4] Number of Checkpoints Taken: 2
 [Manager] Instance 4 Terminated
   [Thread 4] Unread Msg:Thread Exiting
   [Thread 4] Unread Msg:Load Successful
   [Thread 4] Unread Msg:Load Thread Exited
 [Manager] 2 threads remaining
   [Thread 3] Driver Terminated
   [Thread 3] Total Number of Rows Sent: 25
   [Thread 3] Number of Checkpoints Taken: 2
 [Manager] Instance 3 Terminated
   [Thread 3] Unread Msg:Thread Exiting
   [Thread 3] Unread Msg:Load Successful
   [Thread 3] Unread Msg:Load Thread Exited
 [Manager] 1 threads remaining
   [Thread 1] Driver Terminated
   [Thread 1] Total Number of Rows Sent: 25
   [Thread 1] Number of Checkpoints Taken: 2
 [Manager] Instance 1 Terminated
   [Thread 1] Unread Msg:Thread Exiting
   [Thread 1] Unread Msg:Load Successful
   [Thread 1] Unread Msg:Load Thread Exited
 [Manager] 0 threads remaining
 *** Application Complete ***