This document describes how to use the data migration feature of DTS to migrate data from MongoDB to TencentDB for MongoDB.
MongoDB supports heterogeneous migration between replica sets and sharded clusters, that is, it supports four source-target architecture scenarios: replica set – replica set, replica set – sharded cluster, sharded cluster – replica set, sharded cluster – sharded cluster.
Note
When DTS performs full data migration, it will occupy some resources in the source instance, which may increase the load of the source instance and the database pressure. If your database has low configurations, we recommend that you migrate the data during off-peak hours.
To migrate instances over the public network, make sure that the source instance is accessible from the public network.
The source and target databases must meet the requirements for the migration feature and version. For more information, see Databases Supported by Data Migration.
You have completed all the preparations as instructed in Overview.
You need to create a read-only account in the source instance for the migration; otherwise, the precheck will fail.
use admin
db.createUser({
user: "username",
pwd: "password",
roles:
[
{role: "readAnyDatabase", db: "admin"},
{role: "read", db: "local"}
]
})
Constraint
Incremental migration is not supported for self-built single-node instances as they have no oplogs.
Operation Restrictions
Below are the operation restrictions which will impact the success of the task:
Do not modify or delete user information (including username, password, and permissions) and port numbers in the source and target databases.
Do not clear oplogs in the source database.
Do not delete the target database TencetDTSData during data migration.
Manipulate data in the target database with caution during data migration; otherwise, data inconsistency may occur.
As DTS will filter out the DDL operations of the sharded cluster, do not perform DDL operations other than transactions on the source database during shard migration; otherwise, data inconsistency may occur.
If you only perform full data migration, do not write new data into the source instance during migration; otherwise, the data in the source and target databases will be inconsistent. In scenarios with data writes, we recommend that you select full + incremental data migration to ensure data consistency in real time.
Supported SQL Operations
Note
Only replica set migration supports DDL operations, while shard migration will filter out DDL operations (except transactions).
The server where the source database resides must have enough outbound bandwidth; otherwise, the migration speed will be affected.
The user provided by the source database must have the permission to read the database.
The source database cannot have a database named TencetDTSData.
If the source database is in cluster mode, the balancer needs to be disabled before incremental sync.
Oplogs can be obtained from the source database during full + incremental migration.
Requirements for the target database
The target database space must be at least 1.3 times the space of the collections to be migrated in the source database.
The user provided by the target database must have the root permission.
The target database cannot have a database named TencetDTSData.
The target database cannot have collections with the same name as those in the source database.
If the source database is a sharded database, you need to correctly enter the mongos, config server, and mongod node information.
The target database cannot have active businesses; otherwise, an alarm will be reported.
The shard keys of the source and target databases must be the same; otherwise, an alarm will be reported.
Directions
1. Log in to the DTS console, select Data Migration on the left sidebar, and click Create Migration Task.
2. On the Create Migration Task page, select the types, regions, and specifications of the source and target instances and click Buy Now.
Configuration Item
Description
Source Instance Type
Select the source database type, which cannot be changed after purchase. In this scenario, select MongoDB.
Source Instance Region
Select the region where the source database resides. If the source database is a self-built one, select a region nearest to it.
Target Instance Type
Select the target database type, which cannot be changed after purchase. In this scenario, select MongoDB.
Target Instance Region
Select the region where the target database resides.
Specification
Select the specification of the migration link based on your business needs. For the performance and billing details of different specifications, see Billing Overview.
3. On the Set source and target databases page, configure the task, source database, and target database.
Note
You need to create a read-only account in the source instance for the migration; otherwise, the precheck will fail.
Because there are many cross-scenario deployment modes and access types for the source database, and the migration steps for different scenarios are similar, the following only provides configuration examples for typical scenarios. For other scenarios, complete the configurations based on your needs.
Example 1: Migrate a local self-built MongoDB instance (sharded cluster) to a TencentDB instance over Direct Connect
Setting Type
Configuration Item
Description
Task Settings
Task Name
Set a task name that is easy to identify.
Running Mode
Immediate execution: The task will be started immediately after the task verification is passed.
Scheduled execution: The task will be automatically started at the execution time you specify.
Source Database Settings
Source Database Type
The source database type selected during purchase, which cannot be changed.
Region
The source database region selected during purchase, which cannot be changed.
Access Type
Select a type based on your scenario. This scenario takes Direct Connect as an example. For the preparations for different access types, see Overview.
Public Network: The source database can be accessed through a public IP.
Direct Connect: The source database can be interconnected with VPCs through Direct Connect. For more information on Direct Connect, see Direct Connect Documentation.
VPN Access: The source database can be interconnected with VPCs through VPN Connections. For more information on VPN Connections, see VPN Connections Documentation.
Database: The source database is a TencentDB database.
VPC: The source and target databases are both deployed in Tencent Cloud VPCs. For more information on VPC, see Virtual Private Cloud Documentation.
Architecture
Select the architecture of the source database. This scenario takes Cluster migration as an example.
The selected architecture type cannot be modified after the connectivity test is performed; otherwise, an error will be reported.
VPC-Based Direct Connect Gateway
Only VPC-based Direct Connect gateway is supported when the access type is Direct Connect. Confirm the type of the network associated with the gateway.
VPC
Select a VPC and subnet associated with the VPC-based Direct Connect gateway.
Node – mongod
Enter the IP/domain name and port of the mongod node, such as 1xx.3.xx.77:6xx9. For multiple nodes, separate their IPs/domain names and ports by carriage return.
Node – mongos
Enter the IP/domain name and port of the mongos node.
Node – Config Server
Enter the IP/domain name and port of the config server
Authentication Required
Select whether to check the security of username and password in the source database.
Authentication Database
Enter the name of the database to be authenticated, that is, name of the database of the account executing the migration task, such as admin.
Authentication Mechanism
SCRAM-SHA-1 is supported currently
Account & Password Selection
If the three nodes of mongod, mongos, and config server in the source database have the same account and password, select Same account & password; otherwise, select Different accounts & passwords and enter their respective accounts and passwords.
Target Database Settings
Target Database Type
The target database type selected during purchase, which cannot be changed.
Region
The target database region selected during purchase, which cannot be changed.
Access Type
In this scenario, select Database.
Database Instance
Select the instance ID of the target database.
Account
Account of the target database, which must have the required permissions.
Password
Password of the target database account
Example 2: Migrate between two TencentDB instances
Setting Type
Configuration Item
Description
Task Settings
Task Name
Set a task name that is easy to identify.
Running Mode
Immediate execution: The task will be started immediately after the task verification is passed.
Scheduled execution: The task will be automatically started at the execution time you specify.
Tag
Tags are used to manage resources by category in different dimensions. If the existing tags do not meet your requirements, go to the console to create more.
Source Database Settings
Source Database Type
The source database type selected during purchase, which cannot be changed.
Region
The source database region selected during purchase, which cannot be changed.
Access Type
If the source database is a TencentDB instance, select Database. For the preparations for different access types, see Overview.
Public Network: The source database can be accessed through a public IP.
Direct Connect: The source database can be interconnected with VPCs through Direct Connect. For more information on Direct Connect, see Direct Connect Documentation.
VPN Access: The source database can be interconnected with VPCs through VPN Connections. For more information on VPN Connections, see VPN Connections Documentation.
Database: The source database is a TencentDB database.
VPC: The source and target databases are both deployed in Tencent Cloud VPCs. For more information on VPC, see Virtual Private Cloud Documentation.
Cross-/Intra-Account
You need to configure this option when the access type is "Database".
Intra-account: The source and target database instances belong to the same Tencent Cloud root account.
Cross-account: The source and target database instances belong to different Tencent Cloud root accounts. For detailed directions, see Cross-Account TencentDB Instance Migration.
Database Instance
Select the instance ID of source database.
Account
Account of the source TencentDB for MongoDB database, which must have the required permissions.
Password
Password of the source TencentDB for MongoDB database.
Target Database Settings
Target Database Type
The target database type selected during purchase, which cannot be changed.
Region
The target database region selected during purchase, which cannot be changed.
Access Type
In this scenario, select Database.
Database Instance
Select the instance ID of the target database.
Account
Account of the target database, which must have the required permissions.
Password
Password of the target database account
Example 3: Migrate a MongoDB instance (sharded cluster) of other cloud vendors to a TencentDB instance
Setting Type
Configuration Item
Description
Task Settings
Task Name
Set a task name that is easy to identify.
Running Mode
Immediate execution: The task will be started immediately after the task verification is passed.
Scheduled execution: The task will be automatically started at the execution time you specify.
Source Database Settings
Source Database Type
The source database type selected during purchase, which cannot be changed.
Region
The source database region selected during purchase, which cannot be changed.
Access Type
For a third-party cloud database, you can select Public Network generally or select VPN Access, Direct Connect, or CCN based on your actual network conditions. This scenario takes Public Network as an example. For the preparations for different access types, see Overview.
Public Network: The source database can be accessed through a public IP.
Direct Connect: The source database can be interconnected with VPCs through Direct Connect. For more information on Direct Connect, see Direct Connect Documentation.
VPN Access: The source database can be interconnected with VPCs through VPN Connections. For more information on VPN Connections, see VPN Connections Documentation.
Database: The source database is a TencentDB database.
VPC: The source and target databases are both deployed in Tencent Cloud VPCs. For more information on VPC, see Virtual Private Cloud Documentation.
Architecture
Select the architecture of the source database. This scenario takes Cluster migration as an example.
The selected architecture type cannot be modified after the connectivity test is performed; otherwise, an error will be reported.
Node – mongod
Enter the IP/domain name and port of the mongod node, such as 1xx.xx.55.77:6xx9. For multiple nodes, separate their IPs/domain names and ports by carriage return.
Node – mongos
Enter the IP/domain name and port of the mongos node.
Node – Config Server
Enter the IP/domain name and port of the config server
Authentication Required
Select whether to check the security of username and password in the source database.
Authentication Database
Enter the name of the database to be authenticated, that is, name of the database of the account executing the migration task, such as admin.
Authentication Mechanism
SCRAM-SHA-1 is supported currently
Account & Password Selection
If the three nodes of mongod, mongos, and config server in the source database have the same account and password, select Same account & password; otherwise, select Different accounts & passwords and enter their respective accounts and passwords.
Target Database Settings
Target Database Type
The target database type selected during purchase, which cannot be changed.
Region
The target database region selected during purchase, which cannot be changed.
Access Type
In this scenario, select Database.
Database Instance
Select the instance ID of the target database.
Account
Account of the target database, which must have the required permissions.
Password
Password of the target database account
4. Test the connectivity between the source and target databases.
If the connectivity test fails, fix the problem as instructed in Failed Connectivity Test.
5. On the Set migration options and select migration objects page, set the migration options and migration objects. You can select specified databases and collections.
Configuration Item
Description
Migration Type
Select an option as needed.
Full migration: The entire database will be migrated. The data to be migrated only includes the existing data of the source database. The incremental data written to the source database after the task is initiated won't be migrated.
Full + incremental migration: The data to be migrated includes the existing data of the source database and the incremental data written to the source database after the task is initiated. If data needs to be written to the source database during migration and you want to smoothly migrate the data in a non-stop manner, select this option.
Consistency Check
If Full + incremental migration is selected, you can perform a data consistency check to carefully compare the data in the source and target databases after migration and determine the cutover time.
If Full check is selected, DTS will automatically trigger a data consistency check task when the migration task enters the "incremental sync" stage, in which the time lag between the target and the source databases is 0 seconds and the data gap between them is 0 MB.
If Full check is not selected, you can manually trigger a check task when the task enters the incremental sync stage. For more information, see Creating Data Consistency Check Task.
Migration Object
Entire instance: Migrate the entire database instance, including the metadata definitions of roles and users but excluding system databases such as system objects in PostgreSQL.
Specify object: Migrate specified objects.
Specify object
Select the objects to be migrated in the Source Database Object box and move them to the Selected Object box.
6. On the Verify task page, complete the precheck and click Start Task.
If the verification fails, troubleshoot as instructed in Database Connection Check](https://www.tencentcloud.com/document/product/571/42552!73c84487192085ccd5cece97b25dbfcf) and initiate the verification again.
Failed: It indicates that a check item fails and the task is interrupted. You need to fix the problem and run the verification task again.
Alarm: It indicates that a check item doesn't completely meet the requirements, and the task can be continued, but the business will be affected. You need to assess whether to ignore the alarm or fix the problem and continue the task based on the alarm message.
7. Return to the migration task list. After the incremental sync is 100% complete, click Complete in the Operation column to complete the migration task.
Select Full migration: The task will stop automatically after it is completed.
Select Full + incremental migration: After full migration is completed, the migration task will automatically enter the incremental data sync stage, and the incremental sync will not stop automatically. You need to click Complete to manually stop it. Click Complete when the time lag between the source and target databases is 0 seconds.
8. (Optional) If you want to view/delete a task or perform other operations, click the task and select the target operation in the Operation column. For more information, see Viewing Task.
9. After the migration task status becomes Task successful, you can formally cut over the business. For more information, see Cutover Description.
Yes
No
Was this page helpful?