- Release Notes and Announcements
- Product Introduction
- Purchase Guide
- Getting Started
- Preparations
- Business Evaluation
- Network Preparation
- DTS Service Permission Preparation
- Database and Permission Preparation
- Configuring Binlog in Self-Built MySQL
- Data Migration
- Databases Supported by Data Migration
- Cross-Account TencentDB Instance Migration
- Migration to MySQL Series
- Migrating to MySQL
- Migrating to MariaDB
- Migrating to TDSQL-C for MySQL
- Migrating to TDSQL MySQL
- Migrating to TDSQL TDStore
- Advanced Operations for MySQL Series Migration
- Processing for MySQL Series Pre-Check Failure
- Database Connection Check
- Peripheral Check
- Version Check
- Source Instance Permission Check
- Partial Database Parameter Check
- Target Instance Permission Check
- Target Database Content Conflict Check
- Target Database Space Check
- Binlog Parameter Check
- Foreign Key Dependency Check
- View Check
- Advanced Object Check
- Warning Item Check
- TDSQL Proxy Check
- TDSQL Kernel Check
- DDL Ring Linkage Synchronization Detection for Same Database and Table Objects
- DDL Synchronization Conflict Detection for the Same Target and Same Database and Table Object
- Offline Migration of MySQL Data
- Migrating to PostgreSQL
- Migration from PostgreSQL to TencentDB for PostgreSQL
- Migration from TDSQL-C PostgreSQL to TencentDB for PostgreSQL
- PostgreSQL Physical Migration
- Processing for PostgreSQL Series Pre-Check Failure
- Database Connection Check
- Object Dependency Check
- Version Check
- Source or Target Instance Permission Check
- Incremental Migration Pre Check
- Structural Compatibility Check
- Structural Conflict Check
- Account Conflict Check
- Parameter Configuration Conflict Check
- Plugin Compatibility Check
- Multi Task Conflict Detection
- Long Transaction Check
- Migrating to MongoDB
- Migration from MongoDB to MongoDB
- Fix for Verification Failure
- MongoDB Connection Check
- Database/Table Content Conflict Check
- Source Database Node Role Check
- Oplog Check
- Source/Target Database Account Permission Check
- Database Version Check
- Database Capacity Check
- Target Database Load Check
- ShardKey Check
- Source Database Balancer Check
- Time Series Collection Verification
- Compression Algorithm Verification
- MongoDB Data Consistency Verification
- Migrating to SQL Server
- Migrating to TencentDB for Redis
- Migrating to KeeWiDB
- Task Management
- Task Status Description
- Viewing Task
- Configuring Task
- Verifying Task
- Start Task
- Retrying Task (MySQL)
- Retrying Task (Redis)
- Speed Limit (MySQL)
- Speed Limit (Redis)
- Complete Task
- Terminating Task
- Deleting Task
- Pausing Task
- Resuming Task
- Changing Configuration
- Terminating Task
- Deisolating Task
- Deleting Task
- Managing Tag
- Creating a Similar Task
- Viewing Log
- Data Sync
- Databases Supported by Data Sync
- Cross-Account TencentDB Instance Sync
- Sync to MySQL series
- Sync to MySQL
- Sync to MariaDB
- Sync to TDSQL-C for MySQL
- Sync to TDSQL for MySQL
- Sync to TDSQL TDStore
- MySQL Series Advanced Sync Operations
- MySQL series Pre Validation Failed Processing
- Database Connection Check
- Peripheral Check
- Version Check
- Source Instance Permission Check
- Partial Database Parameter Check
- Target Database Content Conflict Check
- Target Database Space Check
- Binlog Parameter Check
- Foreign Key Dependency Check
- View Check
- Advanced Object Check
- Warning Item Check
- TDSQL Proxy Check
- TDSQL kernel check
- DLL Ring Sync Check for Single Database/Table Object
- DLL Sync Conflict Check for Single Database/Table Object in the Same Target
- Synchronize to PostgreSQL
- Synchronization from PostgreSQL to PostgreSQL
- PostgreSQL Pre Validation does not Pass Processing
- Database Connection Check
- Object Dependency Check
- Version Check
- Source or Target Instance Permission Check
- Incremental Migration Pre Check
- Structural Compatibility Check
- Structural Conflict Check
- Account Conflict Check
- Parameter Configuration Conflict Check
- Plugin Compatibility Check
- Primary Key Dependency Check
- Check the Key Parameters of the Instance
- Multi Task Conflict Detection
- Synchronize to Kafka
- Task Management
- Task Status Description
- Viewing Task
- Configuring Task
- Verifying Task
- Starting Task
- Retrying Task (MySQL)
- Speed Limit
- Stopping Task
- Modify Sync configuration
- Pausing Task
- Resuming Task
- Resetting Task
- Deleting Task
- Changing Configuration
- Pay-as-You-Go to Monthly Subscription
- Terminating/Returning Instance
- Renewing Task
- Deisolating Task
- Deleting Instance
- Creating a Similar Task
- Viewing Log
- Deleting Task
- Data Subscription (Kafka Edition)
- Databases Supported by Data Subscription
- MySQL series Data Subscription
- Creating MySQL or TDSQL for MySQL Data Subscription
- Creating TDSQL for MySQL Data Subscription
- Creating MariaDB/Percona Data Subscription
- Consuming MySQL Data
- Advanced Subscription Operations
- Fix for Verification Failure
- Data Subscription for TDSQL PostgreSQL
- MongoDB Data Subscription
- Task Management
- Consumption Management
- Fix for Verification Failure
- Cutover Description
- Monitoring and Alarms
- Ops Management
- Practical Tutorial
- Synchronizing Local Database to the Cloud
- Creating Two-Way Sync Data Structure
- Creating Many-to-One Sync Data Structure
- Creating Multi-Site Active-Active IDC Architecture
- Selecting Data Sync Conflict Resolution Policy
- Using CLB as Proxy for Cross-Account Database Migration
- Migrating Self-Built Databases to Tencent Cloud Databases via CCN
- Best Practices for DTS Performance Tuning
- FAQs
- Error Handling
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- (NewDTS) Data Migration APIs
- DescribeMigrationJobs
- ModifyMigrationJob
- CompleteMigrateJob
- DescribeMigrationDetail
- CreateMigrationService
- CreateMigrateCheckJob
- DescribeMigrationCheckJob
- SkipCheckItem
- StartMigrateJob
- StopMigrateJob
- ResumeMigrateJob
- RecoverMigrateJob
- ModifyMigrateName
- ModifyMigrateJobSpec
- IsolateMigrateJob
- DestroyMigrateJob
- DescribeMigrateDBInstances
- PauseMigrateJob
- ContinueMigrateJob
- Data Sync APIs
- DescribeSyncJobs
- ModifySyncJobConfig
- CreateModifyCheckSyncJob
- DescribeModifyCheckSyncJobResult
- StartModifySyncJob
- CreateSyncJob
- ConfigureSyncJob
- CreateCheckSyncJob
- SkipSyncCheckItem
- DescribeCheckSyncJobResult
- StartSyncJob
- StopSyncJob
- ResumeSyncJob
- ResizeSyncJob
- RecoverSyncJob
- IsolateSyncJob
- DestroySyncJob
- PauseSyncJob
- ContinueSyncJob
- Data Consistency Check APIs
- Data Types
- Error Codes
- DTS API 2018-03-30
- Service Agreement
- Release Notes and Announcements
- Product Introduction
- Purchase Guide
- Getting Started
- Preparations
- Business Evaluation
- Network Preparation
- DTS Service Permission Preparation
- Database and Permission Preparation
- Configuring Binlog in Self-Built MySQL
- Data Migration
- Databases Supported by Data Migration
- Cross-Account TencentDB Instance Migration
- Migration to MySQL Series
- Migrating to MySQL
- Migrating to MariaDB
- Migrating to TDSQL-C for MySQL
- Migrating to TDSQL MySQL
- Migrating to TDSQL TDStore
- Advanced Operations for MySQL Series Migration
- Processing for MySQL Series Pre-Check Failure
- Database Connection Check
- Peripheral Check
- Version Check
- Source Instance Permission Check
- Partial Database Parameter Check
- Target Instance Permission Check
- Target Database Content Conflict Check
- Target Database Space Check
- Binlog Parameter Check
- Foreign Key Dependency Check
- View Check
- Advanced Object Check
- Warning Item Check
- TDSQL Proxy Check
- TDSQL Kernel Check
- DDL Ring Linkage Synchronization Detection for Same Database and Table Objects
- DDL Synchronization Conflict Detection for the Same Target and Same Database and Table Object
- Offline Migration of MySQL Data
- Migrating to PostgreSQL
- Migration from PostgreSQL to TencentDB for PostgreSQL
- Migration from TDSQL-C PostgreSQL to TencentDB for PostgreSQL
- PostgreSQL Physical Migration
- Processing for PostgreSQL Series Pre-Check Failure
- Database Connection Check
- Object Dependency Check
- Version Check
- Source or Target Instance Permission Check
- Incremental Migration Pre Check
- Structural Compatibility Check
- Structural Conflict Check
- Account Conflict Check
- Parameter Configuration Conflict Check
- Plugin Compatibility Check
- Multi Task Conflict Detection
- Long Transaction Check
- Migrating to MongoDB
- Migration from MongoDB to MongoDB
- Fix for Verification Failure
- MongoDB Connection Check
- Database/Table Content Conflict Check
- Source Database Node Role Check
- Oplog Check
- Source/Target Database Account Permission Check
- Database Version Check
- Database Capacity Check
- Target Database Load Check
- ShardKey Check
- Source Database Balancer Check
- Time Series Collection Verification
- Compression Algorithm Verification
- MongoDB Data Consistency Verification
- Migrating to SQL Server
- Migrating to TencentDB for Redis
- Migrating to KeeWiDB
- Task Management
- Task Status Description
- Viewing Task
- Configuring Task
- Verifying Task
- Start Task
- Retrying Task (MySQL)
- Retrying Task (Redis)
- Speed Limit (MySQL)
- Speed Limit (Redis)
- Complete Task
- Terminating Task
- Deleting Task
- Pausing Task
- Resuming Task
- Changing Configuration
- Terminating Task
- Deisolating Task
- Deleting Task
- Managing Tag
- Creating a Similar Task
- Viewing Log
- Data Sync
- Databases Supported by Data Sync
- Cross-Account TencentDB Instance Sync
- Sync to MySQL series
- Sync to MySQL
- Sync to MariaDB
- Sync to TDSQL-C for MySQL
- Sync to TDSQL for MySQL
- Sync to TDSQL TDStore
- MySQL Series Advanced Sync Operations
- MySQL series Pre Validation Failed Processing
- Database Connection Check
- Peripheral Check
- Version Check
- Source Instance Permission Check
- Partial Database Parameter Check
- Target Database Content Conflict Check
- Target Database Space Check
- Binlog Parameter Check
- Foreign Key Dependency Check
- View Check
- Advanced Object Check
- Warning Item Check
- TDSQL Proxy Check
- TDSQL kernel check
- DLL Ring Sync Check for Single Database/Table Object
- DLL Sync Conflict Check for Single Database/Table Object in the Same Target
- Synchronize to PostgreSQL
- Synchronization from PostgreSQL to PostgreSQL
- PostgreSQL Pre Validation does not Pass Processing
- Database Connection Check
- Object Dependency Check
- Version Check
- Source or Target Instance Permission Check
- Incremental Migration Pre Check
- Structural Compatibility Check
- Structural Conflict Check
- Account Conflict Check
- Parameter Configuration Conflict Check
- Plugin Compatibility Check
- Primary Key Dependency Check
- Check the Key Parameters of the Instance
- Multi Task Conflict Detection
- Synchronize to Kafka
- Task Management
- Task Status Description
- Viewing Task
- Configuring Task
- Verifying Task
- Starting Task
- Retrying Task (MySQL)
- Speed Limit
- Stopping Task
- Modify Sync configuration
- Pausing Task
- Resuming Task
- Resetting Task
- Deleting Task
- Changing Configuration
- Pay-as-You-Go to Monthly Subscription
- Terminating/Returning Instance
- Renewing Task
- Deisolating Task
- Deleting Instance
- Creating a Similar Task
- Viewing Log
- Deleting Task
- Data Subscription (Kafka Edition)
- Databases Supported by Data Subscription
- MySQL series Data Subscription
- Creating MySQL or TDSQL for MySQL Data Subscription
- Creating TDSQL for MySQL Data Subscription
- Creating MariaDB/Percona Data Subscription
- Consuming MySQL Data
- Advanced Subscription Operations
- Fix for Verification Failure
- Data Subscription for TDSQL PostgreSQL
- MongoDB Data Subscription
- Task Management
- Consumption Management
- Fix for Verification Failure
- Cutover Description
- Monitoring and Alarms
- Ops Management
- Practical Tutorial
- Synchronizing Local Database to the Cloud
- Creating Two-Way Sync Data Structure
- Creating Many-to-One Sync Data Structure
- Creating Multi-Site Active-Active IDC Architecture
- Selecting Data Sync Conflict Resolution Policy
- Using CLB as Proxy for Cross-Account Database Migration
- Migrating Self-Built Databases to Tencent Cloud Databases via CCN
- Best Practices for DTS Performance Tuning
- FAQs
- Error Handling
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- (NewDTS) Data Migration APIs
- DescribeMigrationJobs
- ModifyMigrationJob
- CompleteMigrateJob
- DescribeMigrationDetail
- CreateMigrationService
- CreateMigrateCheckJob
- DescribeMigrationCheckJob
- SkipCheckItem
- StartMigrateJob
- StopMigrateJob
- ResumeMigrateJob
- RecoverMigrateJob
- ModifyMigrateName
- ModifyMigrateJobSpec
- IsolateMigrateJob
- DestroyMigrateJob
- DescribeMigrateDBInstances
- PauseMigrateJob
- ContinueMigrateJob
- Data Sync APIs
- DescribeSyncJobs
- ModifySyncJobConfig
- CreateModifyCheckSyncJob
- DescribeModifyCheckSyncJobResult
- StartModifySyncJob
- CreateSyncJob
- ConfigureSyncJob
- CreateCheckSyncJob
- SkipSyncCheckItem
- DescribeCheckSyncJobResult
- StartSyncJob
- StopSyncJob
- ResumeSyncJob
- ResizeSyncJob
- RecoverSyncJob
- IsolateSyncJob
- DestroySyncJob
- PauseSyncJob
- ContinueSyncJob
- Data Consistency Check APIs
- Data Types
- Error Codes
- DTS API 2018-03-30
- Service Agreement
Overview
In the DTS data migration task configuration, or when the task is running, the task concurrency number and RPS can be flexibly adjusted based on the data write volume of the source database, and the resource configuration of both the source and target databases.
Note:
This document only provides guidelines for operating DTS performance parameters. To comprehensively enhance the DTS transmission performance, it is necessary to analyze the load and network conditions of the source/target, etc.
Application Scope
Data migration link between MySQL/MariaDB/Percona/TDSQL-C MySQL/TDSQL MySQL.
Notes
1. You can only adjust the speed during the task configuration process, or when the task status is Running.
2. When reducing the task speed, please observe the delay data volume and delay time. DTS by default only caches the intermediate data for 7 days. A long-term large delay volume may cause the intermediate cache data to expire and the task to fail completely.
3. Adjusting the speed during the full export stage will cause the task to restart, and a partial restart may require starting the sync all over again, as follows.
MySQL series data migration tasks default to lock-free sync. In lock-free sync scenarios, during the structure export stage and the non-primary key table export stage, restarting the task requires starting all over again; and during the primary key table export stage, restarting the task will resume sync, without the need to start all over again.
3.1 Restart during full export stage
During the structure export stage and the non-primary key table export stage, restarting the task requires starting all over again;
During the primary key table export stage: If the time from the start of sync (X) to the time the retry operation is performed (Y) is within 48 hours (including 48 hours), breakpoint resume is supported. If it exceeds 48 hours, the task needs to start all over again. Once the task retries or automatically restarts and begins sync from scratch, the time X will be recalculated.
3.2 Restart during full import and incremental sync stages: support breakpoint resume, which does not affect data transfer.
Adjustment Principles
The default number of concurrent threads for a DTS task's full export is 8, which generally does not need to be modified. If adjustment is necessary, please combine monitoring indicators and source/target database configurations, and considering the precondition of not affecting the performance of the source/target database, enhance the DTS transfer speed to the greatest extent.
DTS transfer performance is related to the configuration of the source/target database, DTS task specifications, network latency, network bandwidth, etc. Therefore, when improving DTS transfer performance, simply increasing the number of threads or RPSes may not be effective. It is necessary to analyze where the performance bottleneck lies and solve the specific problem accordingly.
Task Stage | Adjust Scenario | Adjustment Plan | Adjustment Result |
Task Configuration Stage | Source database configuration is too low to handle the increased load from DTS, resulting in the source database crashing after the DTS task starts. | Reduce the number of export threads for the source database to lower the export RPS. | Valid. |
| Target database configuration is too low to handle the increased load from DTS, resulting in the target database crashing after the DTS task starts. | Reduce the number of import threads for the target database to lower the import RPS. | Valid. |
Full Volume Stage | Source/target database configuration is high, yet DTS full volume transfer speed is slow | If the source database configuration is high, increase the number of export threads for the source database If the target database configuration is high, increase the number of import threads for the target database. The default RPS is very large in the full volume stage, and generally, there's no need to adjust the RPS to increase the transfer speed. | It may not always be effective. If adjustments do not take effect, please investigate factors such as network latency and bandwidth, as these can also affect the transfer performance. |
| Target database configuration is low, and DTS writes to the target database too quickly, impacting the target database load (e.g., high CPU occupancy rate) | Reduce the number of import threads for the target database to lower the import RPS. | Valid. |
Incremental Stage | Source database configuration is high, source database write data volume is large, and network latency is high | Check if the number of RPSes has reached the upper limit. Different specifications of DTS tasks correspond to different RPS upper limits. If the number of RPSes has reached the upper limit, first upgrade the DTS task specifications (through Configuration Adjustment operation). | If the upgrade task does not take effect, please investigate factors such as network latency and bandwidth, as these can also affect the transfer performance. |
| Target database configuration is low, and DTS writes to the target database too quickly, impacting the target database load (e.g., high CPU occupancy rate) | 1. Reduce the DTS task specification. 2. Decrease the concurrent thread count for importing into the target database. | Valid. |
| The target database configuration is low, DTS has too many connections, and the target database reaches the thread limit | Reduce the number of threads for target database import. | Valid. |
| DTS writes to the target database, causing lock contention with the database's own writes (e.g., lock wait timeout error) | Reduce the number of threads for importing into the target database to minimize the lock competition. | Valid. |
Speed Adjustment in Task Configuration
During the initial configuration stage of the sync task, in the Advanced Options on the Set sync options and objects page, you can set the transfer speed parameters.
Speed Adjustment When the Task Is Running
You can adjust the speed during the full data sync stage by adjusting the speed parameters for both full and incremental (full export thread count and RPS, full import thread count and RPS, and incremental import thread count). During the incremental data sync stage, only the incremental speed parameters (incremental import thread count) can be modified, with specific operations as follows.
1. Log in to the DTS console, choose Data Sync page in the left sidebar, and select the specified sync task.
2. Adjust the speed during the full data sync stage.
In the Operation column, select More > Limit Speed.
Adjust the speed parameters in the pop-up interface, and click OK after completion.
3. Adjust the speed during the incremental data sync stage.
In the Operation column, select More > Limit Speed.
Yes
No
Was this page helpful?