- Release Notes and Announcements
- Product Introduction
- Purchase Guide
- Getting Started
- Developer Guide
- Operation Guide
- Practical Tutorial
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Message APIs
- Cluster APIs
- Namespace APIs
- Topic APIs
- CMQ Message APIs
- CMQ Management APIs
- UnbindCmqDeadLetter
- RewindCmqQueue
- ModifyCmqTopicAttribute
- ModifyCmqSubscriptionAttribute
- ModifyCmqQueueAttribute
- DescribeCmqTopics
- DescribeCmqTopicDetail
- DescribeCmqSubscriptionDetail
- DescribeCmqQueues
- DescribeCmqQueueDetail
- DeleteCmqTopic
- DeleteCmqSubscribe
- DeleteCmqQueue
- CreateCmqTopic
- CreateCmqSubscribe
- CreateCmqQueue
- Other APIs
- Environment Role APIs
- TDMQ for RabbitMQ APIs
- RocketMQ APIs
- DescribeRocketMQVipInstanceDetail
- DescribeRocketMQMsg
- ModifyRocketMQTopic
- ModifyRocketMQNamespace
- ModifyRocketMQGroup
- ModifyRocketMQCluster
- DescribeRocketMQTopics
- DescribeRocketMQNamespaces
- DescribeRocketMQGroups
- DescribeRocketMQClusters
- DescribeRocketMQCluster
- DeleteRocketMQTopic
- DeleteRocketMQGroup
- DeleteRocketMQCluster
- CreateRocketMQTopic
- CreateRocketMQNamespace
- CreateRocketMQGroup
- CreateRocketMQCluster
- TDMQ for RocketMQ APIs
- Production and Consumption APIs
- Data Types
- Error Codes
- SDK Documentation
- FAQs
- TDMQ Policy
- Service Level Agreement
- General References
- Contact Us
- Glossary
- Release Notes and Announcements
- Product Introduction
- Purchase Guide
- Getting Started
- Developer Guide
- Operation Guide
- Practical Tutorial
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Message APIs
- Cluster APIs
- Namespace APIs
- Topic APIs
- CMQ Message APIs
- CMQ Management APIs
- UnbindCmqDeadLetter
- RewindCmqQueue
- ModifyCmqTopicAttribute
- ModifyCmqSubscriptionAttribute
- ModifyCmqQueueAttribute
- DescribeCmqTopics
- DescribeCmqTopicDetail
- DescribeCmqSubscriptionDetail
- DescribeCmqQueues
- DescribeCmqQueueDetail
- DeleteCmqTopic
- DeleteCmqSubscribe
- DeleteCmqQueue
- CreateCmqTopic
- CreateCmqSubscribe
- CreateCmqQueue
- Other APIs
- Environment Role APIs
- TDMQ for RabbitMQ APIs
- RocketMQ APIs
- DescribeRocketMQVipInstanceDetail
- DescribeRocketMQMsg
- ModifyRocketMQTopic
- ModifyRocketMQNamespace
- ModifyRocketMQGroup
- ModifyRocketMQCluster
- DescribeRocketMQTopics
- DescribeRocketMQNamespaces
- DescribeRocketMQGroups
- DescribeRocketMQClusters
- DescribeRocketMQCluster
- DeleteRocketMQTopic
- DeleteRocketMQGroup
- DeleteRocketMQCluster
- CreateRocketMQTopic
- CreateRocketMQNamespace
- CreateRocketMQGroup
- CreateRocketMQCluster
- TDMQ for RocketMQ APIs
- Production and Consumption APIs
- Data Types
- Error Codes
- SDK Documentation
- FAQs
- TDMQ Policy
- Service Level Agreement
- General References
- Contact Us
- Glossary
Cross-Region Replication/Feature Description
Last updated: 2024-12-03 10:11:09
Overview
Scenario 1: Cross-Region Disaster Recovery
Pulsar Professional Edition supports deployment across availability zones (AZs). When purchasing a Pulsar cluster in a region with three or more AZs, you can choose up to three AZs to deploy cross-AZ instances. The instance's partition replicas are enforced to distribute across nodes in different AZs, ensuring service continuity even if a single AZ becomes unavailable. However, this 3-AZ deployment within the same city does not meet the disaster recovery requirements of financial clients. They require cross-region disaster recovery to quickly switch to a backup region in the event of a region-wide disaster, ensuring business continuity.
Scenario 2: Global Data Archiving
For a global team, business systems may span multiple regions worldwide, requiring cross-region data transmission and centralized data archiving. Data generated from several major cities worldwide can be transmitted via cross-region replication to a specific regional data center for unified archiving. Since these regions may have Topics with identical names, it is necessary to merge same-named Topics from multiple regions into a single Topic in the target cluster.
To address the above scenarios, TDMQ for Apache Pulsar Professional Edition, based on the GEO Replication solution, enables cross-region cluster replication. This supports use cases such as disaster recovery, global data archiving, and cross-region consumption.
How It Works
When Pulsar enables cross-region replication, a Replicator is initiated in the cluster. For details, see Understanding Pulsar's Cross-region Replication. Taking replication from the Shanghai region to the Beijing region as an example (as shown in the diagram):
The Pulsar cluster in the Shanghai region runs the Replicator component. Within this component, a Producer-R is initiated and bound to the Topic1 of the counterpart cluster in the Beijing region. This replicator sends message data to the Beijing IDC in the role of a producer.
Note:
Messages produced in the Shanghai cluster are first persisted locally within the cluster and then asynchronously forwarded to the Beijing cluster.
The Producer-R in the Replicator of the Shanghai cluster is independent of the client Producer1 in the same cluster. The cluster address configured for Producer-R points to the Beijing cluster.
Replication Process Description
1. Producer1 produces a message to Topic1 in the Shanghai cluster;
2. The Shanghai cluster persists the message to BookKeeper;
3. Upon successful persistence in BookKeeper, the message is pushed to the Cursor of the Replicator;
4. The Cursor in the Replicator forwards the message to Topic1 in the Beijing cluster via Producer-R;
5. Once Topic1 in the Beijing cluster is successfully written to BookKeeper, an ACK is sent back to the Cursor of the Replicator in the Shanghai cluster. Upon receiving the ACK, the Cursor in the Shanghai IDC pushes the next message through Producer-R.
Operation Guide
1. Create a professional cluster in the target replication region. On the cluster purchase page, enable the Cross-region replication switch and select the source data cluster;
2. Configure the metadata synchronization linkage for the cluster through the console:
Replication linkage name: Define a name for the synchronization linkage.
Linkage type: Select message level.
Source cluster selection: Select the Pulsar cluster to serve as the data source.
Target cluster selection: Select the pre-created target cluster in a different region. Only clusters with the same Cluster ID will be displayed.
Replication level: Supports three levels: cluster, namespace, and topic.
Cluster level: Suitable for cluster-wide replication.
Namespace level: Ideal for scenarios where clusters in different regions are both active, with namespaces distributed across regions.
Topic level: The smallest granularity for cross-region replication.
3. Once created, the message-level replication tasks will appear on the monitoring page, where you can review real-time replication rates, message backlogs, and other metrics.
Billing Description
Cross-region replication tasks at the message level will incur fees, while metadata-level replication will not. The console provides relevant monitoring metrics to assist with observation.
For detailed billing information on cross-region message-level replication, see Pro Cluster Billing.
Usage Restrictions
1. Feature Support Scope
This feature is supported only in professional clusters.
2. Linkage Replication Direction
Currently, cross-region replication, whether at the message level or metadata level, is unidirectional.
3. Regarding Replication Objects
For a single object, only one synchronization task can be created, even at higher hierarchical levels.
Topics that are part of a synchronization linkage cannot be deleted unless the synchronization linkage is removed first.
4. Overwrite Issues
For existing resources with the same name but different configurations, such as partition count or TTL attributes, the target cluster will not overwrite the configurations after creating a replication task.
Yes
No
Was this page helpful?