Cross-Cluster Message Replication

Recent Pages

Cross-Cluster Message Replication

Last updated: 2024-10-29 15:36:59

Overview
TDMQ for RocketMQ supports synchronizing messages between clusters in the same or different regions, between 4.x or 5.x clusters, or between a 4.x cluster and a 5.x cluster. You can copy messages from a Topic in cluster A to a Topic in cluster B. During this process, TDMQ for RocketMQ supports filtering messages based on specific conditions (such as tag or SQL expression), starting and stopping the replication task at any time, and monitoring the replication task progress and status.
Billing Rules
The cross-cluster message replication feature is currently free. Tencent Cloud will notify you multiple times one month before official billing through internal messages, SMS messages, and emails.
Directions
Creating Tasks
Enter the Cross-cluster Replication page, click Create New Task at the top of the page, and fill in the following fields as required:
Task Name: The name can contain no more than 200 characters consisting of only Chinese characters, digits, letters, hyphens (-), and underscores (_).
Source Topic: Select the region, cluster, and Topic from the drop-down lists. If the required cluster or Topic does not exists, create one on the cluster list page.
Target Topic: Select the region, cluster, and Topic from the drop-down lists. If the required cluster or Topic does not exists, create one on the cluster list page.
Filter Type: Filtering by tag and filtering by SQL are supported.
Reset Method: Replication can start from the latest offset or the specified time point.
Start Task Now: If this option is enabled, the replication task will start immediately after it is created.
﻿
The task list page will appear after you click Create Task, and the task is considered created after initialization.
The replication task you created is unidirectional. It means that messages will automatically be copied from Topic A to Topic B if you create a replication task from Topic A to Topic B. If you requires bidirectional replication, you need to create another task from Topic B to Topic A.
Viewing Task Details
After a replication task is created, you can view it on the task list page and check its status. You can click Start/Pause in the operation bar to quickly start or pause the task.
You cannot modify the configuration of ongoing replication tasks. If you want to modify the configuration, pause the task first and click Edit in the operation bar. Alternatively, you can enter the task details page and click Edit in the upper right corner of the Basic Info area to modify the task configuration.
You can click the task name to enter the task details page and view detailed configuration, such as the filter type and replication start offset. In the Monitoring area, you can view the real-time monitoring data of the current message replication task, such as the message replication TPS (count/s), number of message replication failures, and message sync delay.
﻿
Handling Exceptions
Under normal circumstances, the status bar will display either "Running" or "Paused". If the status is "Failed to Start", you need to check the task operation status and detailed configuration, such as the SQL expression. You can hover over the status text to view the specific reason for the failure.
﻿
If the task fails, you can click Edit in the operation bar to modify the task configuration. You can also enter the task details page and click Edit in the upper right corner in the Basic Info area.