The Teradata Access Module for Kafka provides Failover support for replicated Topics in a Multi-Broker Kafka Server.
- broker0_ip_address:portno represents Broker 0
- broker1_ip_address:portno represents Broker 1
- broker2_ip_address:portno represents Broker 2
There is a replicated Topic in the Multi-Broker Kafka Server, as shown in the following example.
Topic: replicated_topic PartitionCount:3 ReplicationFactor:3 Configs: Topic: replicated_topic Partition: 0 Leader: 0 Replicas: 0,1,2 Isr: 0,1,2 Topic: replicated_topic Partition: 1 Leader: 1 Replicas: 1,2,0 Isr: 1,2,0 Topic: replicated_topic Partition: 2 Leader: 2 Replicas: 2,0,1 Isr: 2,0,1
In this example, the Topic name is "replicated_topic" and it has 3 partitions.
- Leader broker for "Partition: 0" is "Broker 0".
- "Broker 0" being the leader broker is responsible to process all the read and write requests on "Partition: 0".
- "Partition: 0" is being replicated in "Broker 1" and "Broker 2".
If "Broker 0" goes down during consumption of messages then the Kafka Server will allocate a new Leader Broker for "Partition: 0" from the other available brokers where "Partition: 0" is being replicated. The Teradata Access Module for Kafka will recognize the new leader broker for "Partition: 0" and will continue consumption of messages from "Partition: 0" instead of returning error.
To use this feature, all the brokers of the replicated Topic need to be specified in the initialization string through the -BROKERS keyword.