tencent cloud

All product documents
TDMQ for RocketMQ
DocumentationTDMQ for RocketMQMigration to CloudFeeling MigrationMetadata Migration
Metadata Migration
Download PDF
Last updated: 2025-01-25 11:59:07
Metadata Migration
Last updated: 2025-01-25 11:59:07
Download PDF

Overview

When using a TDMQ for RocketMQ cluster, you may need to migrate your existing business, for example, from a self-built or third-party RocketMQ cluster to the TDMQ for RocketMQ cluster.
This document describes how to migrate metadata from self-built open-source RocketMQ to TDMQ for RocketMQ.
Note:
Both dedicated clusters and virtual clusters support metadata migration, and the migration steps are the same.

Prerequisites

You have created a cluster advance and a namespace in the console.

Directions

Step 1. Export the metadata file

If you are using self-built open-source RocketMQ, you can export metadata in the following two ways:

Option 1：Exporting using script online

If your RocketMQ server can access the public network, run the following script on your server directly (if there are multiple servers in your RocketMQ cluster, you can run the script on any server as long as the network is interconnected in the cluster).
/bin/bash -c "$(curl -fsSL https://rocketmq-1306598660.cos.ap-guangzhou.myqcloud.com/rocketmq-export.sh)"

Option 2：Executing script locally

If your RocketMQ server has no permission to access the public network, follow the steps below:
1. Download the migration tool.
2. Upload the tool to your self-built RocketMQ cluster (if there are multiple servers in your RocketMQ cluster, you can run the tool on any server as long as the network is interconnected in the cluster).
3. Decompress the tool and enter the directory.
unzip rocketmq-migration.zip
cd rocketmq-migration
4. Run the following command for migration.
// Run this command mainly to use the export tool's file as the executable file
ROCKETMQ_HOME=`pwd`
// Starting export
./bin/export.sh
// Enter the open-source RocketMQ address, such as `localhost:9876`
Enter name server address list:localhost:9876
// Select a cluster to export, such as `DefaultCluster`
Choose a cluster to export:DefaultCluster
// Enter the directory for metadata export. By default, the final metadata is stored in /tmp/rocketmq/config/rocketmq-metadata-export.json
Enter file path to export [default /tmp/rocketmq/export]:

Method 3: Exporting/Importing via .xlsx file

If your metadata comes from another Tencent Cloud RocketMQ cluster, you can directly upload topic and group data by uploading the .xlsx template.
Directly export metadata via the

button at the top right corner of the topic/group list page. The export format of the metadata is an .xlsx file. Then go to the topic/group list page of the new cluster and use the

button at the top right corner to import the data into the specified namespace of the 4.x cluster or the 5.x cluster.
For newly created clusters, you can also edit the metadata in the table according to the format of the aforementioned metadata template .xlsx and upload it. You can get the template on the right side of the Migrate to Cloud > Metadata Import page. When obtaining the template, distinguish between 4.x clusters and 5.x clusters by selecting different types of target cluster.


Step 2. Create a migration task

1. Log in to the TDMQ for RocketMQ console and enter the Migration to Cloud page, enter the migration task list page, and click Create Task to create a corresponding migration task.

2. Select the migration task type:
Import to cluster: Migrate metadata from the self-built cluster to the created cluster. During the "Cluster Migration" task, the migration tool will by default parse the part before "%" in the topic name in the open-source RocketMQ Dashboard as the namespace name, thus creating multiple logically isolated namespaces for you. If there are no namespaces in the self-built cluster, the system will generate a default namespace named "default" for you (only applicable to 4.x exclusive clusters and existing 4.x virtual clusters).
Export to Specified namespace: Only 4.x exclusive clusters and existing 4.x virtual clusters are supported. Migrate metadata from the self-built cluster to a specified namespace. If the original topics do not distinguish between namespaces or business categories, you can take the opportunity of cloud migration to select specific topics and groups during the import process, and specify them into different namespaces to distinguish between businesses or environments.

Note：
5.x clusters and 4.x general clusters support importing role and ACL information.
5.x custers and 4.x general clusters do not have namespaces, so only import to cluster is supported, there is no "Export to specified namespace" option.
3. Upload the metadata file obtained in step 1 and select the topics and groups you want to import.
Note:
Up to 1,000 topics and 1,000 groups can be imported in a single task. Excess data will fail to be imported.


Step 3. Check the task status

After the task is successfully created, enter the task list to view the task status. If there is too much data, the task needs to load for a while. Click View Details to view the specific running status of the task.

If the task status is Some failed or All failed, you can filter causes of the failures in the Task Status column.



Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon