About Using the Synchronization Service - Teradata Data Mover

Teradata Data Mover User Guide

Product
Teradata Data Mover
Release Number
16.10
Published
June 2017
Language
English (United States)
Last Update
2018-03-29
dita:mapPath
kmo1482331935137.ditamap
dita:ditavalPath
ft:empty
dita:id
B035-4101
lifecycle
previous
Product Category
Analytical Ecosystem

If multiple Data Mover servers are installed in your environment, you can use the Data Mover synchronization service to create a backup system that can be used as a temporary master system in case of daemon failure on the original master system.

When using the synchronization service with multiple Data Mover servers, the following terminology applies:
Master server
Refers to the Data Mover server on which the daemon is running.
Slave server
Refers to one or more Data Mover servers on which the daemon is not running. Slave servers can be used as a primary system in case of failure on the master server.
The service synchronizes the slave repository with the master repository through replication. To monitor changes in the repository, triggers installed on repository tables record all values for the INSERT statements into an audit log table. Furthermore, the daemon generates the UPDATE and DELETE SQL statements and inserts them into this table. The data from the table is then read by a service of the master synchronization system and is written to a file. Another master synchronization service transfers the data to the slave system, where it is written to a file by the slave synchronization system. The content of the file is then executed against the slave system repository. When the master server for the daemon fails, you can switch to a slave server and continue to run jobs with minimal delay. The synchronization service should be used only when the master and slave Data Mover repositories have the same hash algorithm. This is necessary for the proper execution of the post-failover steps, to return the designated new master to its original role as slave.

Replication begins for the first time when the synchronization service running on the server that is used for the slave repository connects through a TCP/IP socket to the system that is used for the master repository. When the synchronization service starts, updates to the master repository are replicated automatically on the slave repository.

As part of the failover functionality, when the slave server is unavailable (offline, down, restarting), the master repository continues to log updates. When a connection is re-established with the slave server, all interim updates from the master repository that occurred when the slave server was unavailable are applied to the slave repository.

If the slave is down too long, interim updates will accumulate and may cause a memory issue on the master synchronization service. When a slave node connects to the master, the master synchronization services checks to see if there are too many updates for the slave; if so, the master synchronization service disconnects the slave node and generates the exception "Slave has xxxx bytes sqls to be processed since last disconnection, which exceeds current available memory on the master sync service. Slave node will be disconnected. Please run backup and restore to sync the master and slave nodes." As indicated in the message, run backup and restore to sync the master. For more information, see Synchronizing the Master and Slave Repositories.

Synchronization Service Files
File Name Description
/etc/opt/teradata/datamover/sync.properties Settings that the Data Mover Replication Service uses for synchronizing master with slave repositories.
/opt/teradata/datamover/sync/nn.nn/DMReplication.jar Executable binary file used by the synchronization services.
/opt/teradata/datamover/sync/nn.nn/dmsync Script for starting the synchronization service.

Usage Notes

You can use host name or IP address as values for the master.host and jobstore.host in sync.properties for the master and slave servers. When configuring the servers, use only all host names or all IP addresses. Do not mix host names and IP addresses.

Error Conditions
Scenario Result
The sync.properties file is not located at /etc/opt/teradata/datamover The synchronization service reports an error.
The dmsync or DMReplication.jar file is not located at /opt/teradata/datamover/daemon/nn.nn The synchronization service reports an error.
The slave synchronization service is started before the master synchronization service The synchronization service reports an error.
The master and slave synchronization services are started on the same server The synchronization service reports an error.
The master repository is unavailable (offline, down, restarting) The synchronization service stops all connected slave repositories.
The master repository is restarted after a period of unavailability The synchronization service reports an error. The master sync service must be restarted by running /opt/teradata/datamover/sync/nn.nn/dmsync start, where nn.nn in the path refers to the major and minor version numbers of Data Mover. Previously connected slave services must be restarted individually by running /opt/teradata/datamover/sync/nn.nn/dmsync start for each one.