TDSQL MySQL Offline Single Table Read Node Configuration
Parameters
Description
Data Source
Available TDSQL for MySQL Data Source.
Database
Supports selecting or manually entering the database name to be read
By default, the database bound to the data source is used as the default database. Other databases need to be manually entered.
If the data source network is not connected and the database information cannot be fetched directly, you can manually enter the database name. Data synchronization can still be performed when the Data Integration network is connected.
Table
Supports selecting or manually entering the table name to be read
If the data source network is not connected and the table information cannot be fetched directly, you can manually enter the table name. Data synchronization can still be performed when the Data Integration network is connected.
Split Key
Specify the field used for data sharding. After specifying, concurrent tasks will be initiated for data synchronization, improving data synchronization efficiency. You can use a column in the source data table as the partition key. It is recommended to use the primary key or indexed columns as the partition key. Only integer-type fields are supported.
Filter Conditios(Optional)
In actual business scenarios, data from the current day is usually chosen for synchronization. Specify the where condition as gmt_create>$bizdate. The where condition can effectively carry out business incremental synchronization.
If the WHERE clause is not provided, including missing the key or value, the data synchronization is treated as full data synchronization.
The where condition cannot be specified as limit 10, as it does not comply with TDSQL for MySQL WHERE clause constraints.
Advanced Settings (Optional)
You can configure parameters according to business needs.
TDSQL for MySQL Offline Single Table Write Node Configuration
Parameters
Description
Data Destination
TDSQL for MySQL Data Source to write to.
Database
Supports selection or manual input of the database name to write to
By default, the database bound to the data source is used as the default database. Other databases need to be manually entered.
If the data source network is not connected and the database information cannot be fetched directly, you can manually enter the database name. Data synchronization can still be performed when the Data Integration network is connected.
Table
Supports selection or manual input of the table name to write to
If the data source network is not connected and the table information cannot be fetched directly, you can manually enter the table name. Data synchronization can still be performed when the Data Integration network is connected.
Whether to Clear Table
Before writing to the TDSQL MySQL data table, you can manually choose whether to clear the data table.
Write Mode
TDSQL for MySQL supports three writing modes:
Append: When a primary key/unique index conflict occurs, the conflicting row cannot be written.
Overwrite: When a primary key/unique index conflict occurs, the original row is deleted first, and then the new row is inserted.
On duplicate key: When a primary key/unique index conflict occurs, the new row replaces the specified fields of the existing row.
Batch Submission Size
The record size of one-time batch submission can greatly reduce the number of network interactions between the data synchronization system and TDSQL and improve overall throughput. If the value is set too high, it may cause the data synchronization process to encounter OOM exceptions.
Pre-Executed SQL(Optional)
The SQL statement executed before the synchronization task. Fill in the correct SQL syntax according to the data source type, such as clearing the old data in the table before execution (truncate table tablename).
Post-Executed SQL (Optional)
The SQL statement executed after the synchronization task. Fill in the correct SQL syntax according to the data source type, such as adding a timestamp (alter table tablename add colname timestamp DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP).
