You can view the metric monitoring data in real time to stay up to date with the metric performance of running tasks.
Statistical dimension (data migration: app_id, migratejob_id; data sync: appid, replicationjobid; data subscription: appId, subscribeid)
Statistical period: 60s, 300s.
BPS: Bits per second, indicating the volume of data transferred per second.
RPS: Rows per second, indicating the number of rows transferred per second.
Application Scope
Monitoring metrics can be displayed for the migration and sync links of MySQL/MariaDB/Percona/TDSQL-C for MySQL/TDSQL for MySQL.
Monitoring metrics can be displayed for the subscription links of MySQL/MariaDB/Percona/TDSQL-C for MySQL/TDSQL for MySQL/TDSQL for PostgreSQL
Data Migration
MySQL/MariaDB/Percona/TDSQL-C for MySQL
Metric Category
Metric Name
Parameter
Unit
Description
BPS – Full Stage
Full Export BPS
MigrateDumperBps
MB/s
Volume of data exported from the source instance per second in the full stage
Full Import BPS
MigrateLoaderBps
MB/s
Volume of data imported to the target instance per second in the full stage
BPS – Incremental Stage
Incremental Export BPS
MigrateRiverBps
MB/s
Volume of data exported from the source instance per second in the incremental stage
Incremental Import BPS
MigrateSinkerBps
MB/s
Volume of data imported to the target instance per second in the incremental stage
RPS – Full Stage
Full Export RPS
MigrateDumperRps
Count/s
Number of rows of data exported from the source instance per second in the full stage
Full Import RPS
MigrateLoaderRps
Count/s
Number of rows of data imported to the target instance per second in the full stage
RPS – incremental stage
Source Instance Data Extraction RPS (Incremental Export RPS)
MigrateCaptureRps
Count/s
This metric is only used in the transitory stage, and the "Incremental Export RPS" metric will be finally used.
Incremental Export RPS
MigrateRiverRps
Count/s
Number of rows of data exported from the source instance per second in the incremental stage
Target Instance Data Loading RPS (Incremental Import RPS)
MigrateLoadRps
Count/s
This metric is only used in the transitory stage, and the "Incremental Import RPS" metric will be finally used.
Incremental Import RPS
MigrateSinkerRps
Count/s
Number of rows of data imported to the target instance per second in the incremental stage
Network Latency – Full Stage
Source Instance Network Latency During Full Export
MigrateDumperNetworkLag
ms
Network latency between data export and source instance in the full stage
Target Instance Network Latency During Full Import
MigrateLoaderNetworkLag
ms
Network latency between data import and target instance in the full stage
Network Latency – Incremental Stage
Source Instance Network Latency During Incremental Export
MigrateRiverNetworkLag
ms
Network latency between data export and source instance in the incremental stage
Target Instance Network Latency During Incremental Import
MigrateSinkerNetworkLag
ms
Network latency between data import and target instance in the incremental stage
Data Migration Time Lag
MigrateLag
s
Time lag between source and target instances in the incremental stage
Calculation method: Time lag = the current time of the source instance - the time recorded when the latest source instance binlog event is being executed in the target instance.
Calculation of the data migration time lag depends on the incremental binlogs of the source database. Therefore, if there are no DDL or DML operations performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual migration time lag. If the value is -1, it indicates that the existing data has been migrated, but there is no refresh of the incremental data. In this case, you can refresh the metric by running a SQL statement on the source database to get the correct metric value.
Data Gap in Data Migration
MigrateLagData
MBytes
Data gap between the source and target instances in the incremental stage
Calculation method: Data gap = the file offset of the latest source instance binlog event - the file offset of the latest source instance binlog event which is being executed in the target instance. If the two offsets are in different binlog files, this value will be an estimate.
If there are no DDL or DML operations performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual sync data gap. If the value is -1, it indicates that the existing data has been synced, but there is no refresh of the incremental data.
RPS Utilization
RPS Utilization During Incremental Import
MigrateSinkerRpsUsage
%
RPS utilization = real-time RPS/maximum RPS. The maximum RPS depends on the specification you select when configuring the link.
When the RPS utilization almost reaches 100%, the incremental migration can no longer be accelerated. You need to upgrade the specification for faster migration speed.
When the link is configured with the highest specification, there is no RPS limit, and the RPS may exceed 100%.
TDSQL for MySQL
Currently, only the following incremental migration metrics are supported.
Metric Name
Parameter
Unit
Description
Source Instance Data Extraction RPS
MigrateCaptureRps
Count/s
Number of rows of source instance data read by DTS per second
Target Instance Data Loading RPS
MigrateLoadRps
Count/s
Number of rows of data migrated to the target instance per second
Data Migration Latency
MigrateLag
s
Time lag between source and target instances
Calculation method: Time lag = the current time of the source instance - the time recorded when the latest source instance binlog event is being executed in the target instance.
Calculation of the data migration time lag depends on the incremental binlogs of the source database. Therefore, if there are no DDL or DML operations performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual migration time lag. If the value is -1, it indicates that the existing data has been migrated, but there is no refresh of the incremental data. In this case, you can refresh the metric by running a SQL statement on the source database to get the correct metric value.
Data Gap in Data Migration
MigrateLagData
MBytes
Data gap between the source and target instance
Calculation method: Data gap = the file offset of the latest source instance binlog event - the file offset of the latest source instance binlog event which is being executed in the target instance. If the two offsets are in different binlog files, this value will be an estimate.
If there are no DDL or DML operations performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual sync data gap. If the value is -1, it indicates that the existing data has been migrated, but there is no refresh of the incremental data.
Data Sync
MySQL/MariaDB/Percona/TDSQL-C for MySQL
Metric Category
Metric Name
Parameter
Unit
Description
BPS – Full Stage
Full Export BPS
ReplicationDumperBps
MB/s
Volume of data exported from the source instance per second in the full stage
Full Import BPS
ReplicationLoaderBps
MB/s
Volume of data imported to the target instance per second in the full stage
BPS – Incremental Stage
Incremental Export BPS
ReplicationRiverBps
MB/s
Volume of data exported from the source instance per second in the incremental stage
Incremental Import BPS
ReplicationSinkerBps
MB/s
Volume of data imported to the target instance per second in the incremental stage
RPS – Full Stage
Full Export RPS
ReplicationDumperRps
Count/s
Number of rows of data exported from the source instance per second in the full stage
Full Import RPS
ReplicationLoaderRps
Count/s
Number of rows of data imported to the target instance per second in the full stage
RPS – incremental stage
Source Instance Data Extraction RPS (Incremental Export RPS)
ReplicationCaptureRps
Count/s
This metric is only used in the transitory stage, and the "Incremental Export RPS" metric will be finally used.
Incremental Export RPS
ReplicationRiverRps
Count/s
Number of rows of data exported from the source instance per second in the incremental stage
Target Instance Data Loading RPS (Incremental Import RPS)
ReplicationLoadRps
Count/s
This metric is only used in the transitory stage, and the "Incremental Import RPS" metric will be finally used.
Incremental Import RPS
ReplicationSinkerRps
Count/s
Number of rows of data imported to the target instance per second in the incremental stage
Network Latency – Full Stage
Source Instance Network Latency During Full Export
ReplicationDumperNetworkLag
ms
Network latency between data export and source instance in the full stage
Target Instance Network Latency During Full Import
ReplicationLoaderNetworkLag
ms
Network latency between data import and target instance in the full stage
Network Latency – Incremental Stage
Source Instance Network Latency During Incremental Export
ReplicationRiverNetworkLag
ms
Network latency between data export and source instance in the incremental stage
Target Instance Network Latency During Incremental Import
ReplicationSinkerNetworkLag
ms
Network latency between data import and target instance in the incremental stage
Data Sync Time Lag
DtsReplicationLag
s
Sync time lag between source and target instances in the incremental stage
Calculation method: Time lag = the current time of the source instance - the time recorded when the latest source instance binlog event is being executed in the target instance.
Calculation of the data sync time lag depends on the incremental binlogs of the source database. Therefore, if there are no DDL or DML operations performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual sync time lag. If the value is -1, it indicates that the existing data has been synced, but there is no refresh of the incremental data. In this case, you can refresh the metric by running a SQL statement on the source database to get the correct metric value.
Data Gap in Data Sync
DtsReplicationLagData
MBytes
Sync data gap between the source and target instances in the incremental stage.
Calculation method: Data gap = the file offset of the latest source instance binlog event - the file offset of the latest source instance binlog event which is being executed in the target instance. If the two offsets are in different binlog files, this value will be an estimate.
If there are no DDL or DML operations performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual sync data gap. If the value is -1, it indicates that the existing data has been synced, but there is no refresh of the incremental data.
RPS utilization
RPS Utilization During Incremental Import
ReplicationSinkerRpsUsage
%
RPS utilization = real-time RPS/maximum RPS. The maximum RPS depends on the specification you select when configuring the link.
When the RPS utilization almost reaches 100%, the incremental migration can no longer be accelerated. You need to upgrade the specification for faster migration speed.
When the link is configured with the highest specification, there is no RPS limit, and the RPS may exceed 100%.
TDSQL for MySQL
Currently, only the following incremental sync metrics are supported.
Metric Name
Parameter
Unit
Description
Source Instance Data Extraction RPS
ReplicationCapture_rps
Count/s
Number of rows of source instance data read by DTS per second
Target Instance Data Loading RPS
ReplicationLoadRps
Count/s
Number of rows of data migrated to the target instance per second
Data Sync Time Lag
DtsReplicationLag
s
Time lag between source and target instances
Calculation method: Time lag = the current time of the source instance - the time recorded when the latest source instance binlog event is being executed in the target instance.
Calculation of the data sync time lag depends on the incremental binlogs of the source database. Therefore, if there are no DDL or DML operations performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual sync time lag. If the value is -1, it indicates that the existing data has been migrated, but there is no refresh of the incremental data. In this case, you can refresh the metric by running a SQL statement on the source database to get the correct metric value.
Data Gap in Data Sync
DtsReplicationLagData
MBytes
Data gap between the source and target instance
Calculation method: Data gap = the file offset of the latest source instance binlog event - the file offset of the latest source instance binlog event which is being executed in the target instance. If the two offsets are in different binlog files, this value will be an estimate.
If there are no DDL or DML operations performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual sync data gap. If the value is -1, it indicates that the existing data has been synced, but there is no refresh of the incremental data.
Data Subscription
MySQL/TDSQL-C for MySQL
Metric Category
Metric Name
Parameter
Unit
Description
Data Production
GTID Quantity Gap Between Subscription Service and Source Database
ProducerLag
Count
Gap between the number of GTIDs in the binlog event already parsed by the data subscription service and the number of GTIDs in the latest binlog event generated by the source database.
If there are no DDL or DML operations performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual sync time lag. If the value is -1, it indicates that there is no refresh of the incremental data.
Number of Transactions Parsed per Second
ProducerTps
Count/s
Number of transactions parsed from the source instance binlog per second in the incremental stage
Incremental Export BPS
SubscribeRiverBps
MB/s
Volume of data exported from the source instance per second in the incremental stage
Incremental Import BPS to Kafka
SubscribeSinkerBps
MB/s
Volume of data imported to the built-in Kafka of the subscription service per second in the incremental stage
Data Consumption
Subscription Partition Consumption Gap
SubscribePartitionLag
Count
Gap between the offset of data to be consumed and that of consumed data. The statistical dimensions are "Partition", "Group", and "Task".
Subscription Partition Consumption Time Lag
SubscribePartitionLagTime
s
Time lag between the consumed data and the source instance. The statistical dimensions are "Partition", "Group", and "Task".
MariaDB/Percona/TDSQL for MySQL
Metric Name
Parameter
Unit
Description
GTID Quantity Gap Between Subscription Service and Source Database
ProducerLag
Count
Gap between the number of GTIDs in the binlog event already parsed by the data subscription service and the number of GTIDs in the latest binlog event generated by the source database.
If there are no DDL or DML operations performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual sync time lag. If the value is -1, it indicates that there is no refresh of the incremental data.
Number of Transactions Parsed per Second
ProducerTps
Count/s
Number of transactions parsed from the source instance binlog per second
TDSQL for PostgreSQL
Metric Name
Parameter
Unit
Description
LSN Gap Between Subscription Service and Source Database
ProducerLsnLag
MBytes
LSN gap between the offset of parsed logs and that of the latest logs
Number of Transactions Parsed per Second
ProducerTps
Count/s
Number of transactions parsed from the source instance binlog per second
Viewing Monitoring Metrics
1. Log in to the DTS console and select Data Migration, Data Sync, or Data Subscription on the left sidebar as needed.
2. You can view monitoring metrics in the two ways below:
Option 1: Select the target migration task and click the
icon to enter the monitoring view. You can also click the
icon to view the detailed metric monitoring information.
Option 2: Click the ID of the target migration task to enter the task details page.
Select the Monitoring Data tab to view the metric monitoring data.
3. Select a time range.
You can select a custom time range or select from the available time ranges quickly.
Three time dimensions are supported for comparison: week-over-week, day-over-day, and custom date comparison.
