TDSQL-C MySQL Data Source

TDSQL-C for MySQL offline single table read node configuration
﻿
﻿
﻿
Parameters
Description
Data Source
Available TDSQL-C for MySQL data source.
Database
Supports selecting or manually entering the database name to be read
By default, the database bound to the data source is used as the default database. Other databases need to be manually entered.
If the data source network is not connected and the database information cannot be fetched directly, you can manually enter the database name. Data synchronization can still be performed when the Data Integration network is connected.
Table
Supports selecting or manually entering the table name to be read
In the case of table sharding, you can select or enter multiple table names on the TDSQL-C for MySQL source end, ensuring the structure is consistent across tables.
In the case of table partitioning, the table index range can be configured. For instance, 'table_[0-99]' indicates reading 'table_0', 'table_1', 'table_2' up to 'table_99'; if the numeric suffix of your tables has a consistent length, like 'table_000','table_001','table_002' up to 'table_999', you can configure it as '"table": ["table_00[0-9]", "table_0[10-99]", "table_[100-999]"]'.
If the data source network is not connected and the table information cannot be fetched directly, you can manually enter the table name. Data synchronization can still be performed when the Data Integration network is connected.
Add Sharded Database/Table
Multiple data sources can be created, and corresponding table objects can be added.
Note:
For sharded databases and tables, ensure the schema information of the selected multiple table objects is consistent (including field names and field types). The system defaults to displaying the metadata field information of the first table of the first data source in the data fields module. If fields between multiple tables are inconsistent, it may cause runtime failures.
Split Key
Specify the field used for data sharding. After specifying, concurrent tasks will be initiated for data synchronization, improving data synchronization efficiency. You can use a column in the source data table as the partition key. It is recommended to use the primary key or indexed columns as the partition key. Only integer-type fields are supported.
Filter Conditions (Optional)
In actual business scenarios, data from the current day is usually chosen for synchronization. Specify the where condition as gmt_create>$bizdate. The where condition can effectively carry out business incremental synchronization.
If the WHERE clause is not provided, including missing the key or value, the data synchronization is treated as full data synchronization.
You cannot specify the WHERE condition as limit 10, which does not comply with TDSQL-C for MySQL WHERE clause constraints.
Advanced Settings (Optional)
You can configure parameters according to business needs.
TDSQL-C for MySQL offline single table write node configuration
﻿
﻿
﻿
Parameters
Description
Data Destination
The TDSQL-C for MySQL data source to be written.
Write Type
Single table write: Supports single table writing
Sharding write: Supports one-time writing to sharded TDSQL-C for MySQL tables within the same data source. Ensure all table schema structures are consistent and all target tables exist.
Database
Supports selection or manual input of the database name to write to
By default, the database bound to the data source is used as the default database. Other databases need to be manually entered.
If the data source network is not connected and the database information cannot be fetched directly, you can manually enter the database name. Data synchronization can still be performed when the Data Integration network is connected.
Table
Supports selection or manual input of the table name to write to
If the data source network is not connected and the table information cannot be fetched directly, you can manually enter the table name. Data synchronization can still be performed when the Data Integration network is connected.
Whether to Clear Table
You can manually choose whether to clear the TDSQL-C for MySQL data table before writing to it.
Write Mode
TDSQL-C for MySQL supports three write modes:
Append: When a primary key/unique index conflict occurs, the conflicting rows cannot be written.
Overwrite: When a primary key/unique index conflict occurs, the original row is deleted, and the new row is inserted.
On duplicate key: When a primary key/unique index conflict occurs, the new row replaces the specified fields of the existing row.
Batch Submission Size
The record size of one-time batch submission can greatly reduce the number of network interactions between the data synchronization system and TDSQL and improve overall throughput. If the value is set too high, it may cause the data synchronization process to encounter OOM exceptions.
Pre-Executed SQL (Optional)
The SQL statement executed before the synchronization task. Fill in the correct SQL syntax according to the data source type, such as clearing the old data in the table before execution (truncate table tablename).
Post-Executed SQL (Optional)
The SQL statement executed after the synchronization task. Fill in the correct SQL syntax according to the data source type, such as adding a timestamp (alter table tablename add colname timestamp DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP).
Data type conversion support
Read  
The data types and their conversion relationships supported by TDSQL-C for MySQL read are as follows (when processing TDSQL-C for MySQL, the data types of the TDSQL-C for MySQL data source will be mapped to the data types of the data processing engine):
MySQL Data Types
Internal Types
tinyint, smallint, mediumint, int, bigint,year
Long
float, double, decimal
Double
varchar, char, tinytext, text, mediumtext, longtext,set,json
String
date, datetime, timestamp, time
Date
bit, bool
Boolean
tinyblob, mediumblob, blob, longblob, varbinary
Bytes
Write
The data types and their conversion relationships supported by TDSQL-C for MySQL write are as follows:
Internal Types
MySQL Data Types
Long
tinyint, smallint, mediumint, int, bigint,year
Double
float, double, decimal
String
varchar, char, tinytext, text, mediumtext, longtext,set,json
Date
date, datetime, timestamp, time
Boolean
bit, bool
Bytes
tinyblob, mediumblob, blob, longblob, varbinary
﻿

Was this page helpful?

You can also Contact Sales or Submit a Ticket for help.

Yes

Parameters	Description
Data Source	Available TDSQL-C for MySQL data source.
Database	Supports selecting or manually entering the database name to be read By default, the database bound to the data source is used as the default database. Other databases need to be manually entered. If the data source network is not connected and the database information cannot be fetched directly, you can manually enter the database name. Data synchronization can still be performed when the Data Integration network is connected.
Table	Supports selecting or manually entering the table name to be read In the case of table sharding, you can select or enter multiple table names on the TDSQL-C for MySQL source end, ensuring the structure is consistent across tables. In the case of table partitioning, the table index range can be configured. For instance, 'table_[0-99]' indicates reading 'table_0', 'table_1', 'table_2' up to 'table_99'; if the numeric suffix of your tables has a consistent length, like 'table_000','table_001','table_002' up to 'table_999', you can configure it as '"table": ["table_00[0-9]", "table_0[10-99]", "table_[100-999]"]'. If the data source network is not connected and the table information cannot be fetched directly, you can manually enter the table name. Data synchronization can still be performed when the Data Integration network is connected.
Add Sharded Database/Table	Multiple data sources can be created, and corresponding table objects can be added. Note: For sharded databases and tables, ensure the schema information of the selected multiple table objects is consistent (including field names and field types). The system defaults to displaying the metadata field information of the first table of the first data source in the data fields module. If fields between multiple tables are inconsistent, it may cause runtime failures.
Split Key	Specify the field used for data sharding. After specifying, concurrent tasks will be initiated for data synchronization, improving data synchronization efficiency. You can use a column in the source data table as the partition key. It is recommended to use the primary key or indexed columns as the partition key. Only integer-type fields are supported.
Filter Conditions (Optional)	In actual business scenarios, data from the current day is usually chosen for synchronization. Specify the where condition as gmt_create>$bizdate. The where condition can effectively carry out business incremental synchronization. If the WHERE clause is not provided, including missing the key or value, the data synchronization is treated as full data synchronization. You cannot specify the WHERE condition as limit 10, which does not comply with TDSQL-C for MySQL WHERE clause constraints.
Advanced Settings (Optional)	You can configure parameters according to business needs.

Parameters	Description
Data Destination	The TDSQL-C for MySQL data source to be written.
Write Type	Single table write: Supports single table writing Sharding write: Supports one-time writing to sharded TDSQL-C for MySQL tables within the same data source. Ensure all table schema structures are consistent and all target tables exist.
Database	Supports selection or manual input of the database name to write to By default, the database bound to the data source is used as the default database. Other databases need to be manually entered. If the data source network is not connected and the database information cannot be fetched directly, you can manually enter the database name. Data synchronization can still be performed when the Data Integration network is connected.
Table	Supports selection or manual input of the table name to write to If the data source network is not connected and the table information cannot be fetched directly, you can manually enter the table name. Data synchronization can still be performed when the Data Integration network is connected.
Whether to Clear Table	You can manually choose whether to clear the TDSQL-C for MySQL data table before writing to it.
Write Mode	TDSQL-C for MySQL supports three write modes: Append: When a primary key/unique index conflict occurs, the conflicting rows cannot be written. Overwrite: When a primary key/unique index conflict occurs, the original row is deleted, and the new row is inserted. On duplicate key: When a primary key/unique index conflict occurs, the new row replaces the specified fields of the existing row.
Batch Submission Size	The record size of one-time batch submission can greatly reduce the number of network interactions between the data synchronization system and TDSQL and improve overall throughput. If the value is set too high, it may cause the data synchronization process to encounter OOM exceptions.
Pre-Executed SQL (Optional)	The SQL statement executed before the synchronization task. Fill in the correct SQL syntax according to the data source type, such as clearing the old data in the table before execution (truncate table tablename).
Post-Executed SQL (Optional)	The SQL statement executed after the synchronization task. Fill in the correct SQL syntax according to the data source type, such as adding a timestamp (alter table tablename add colname timestamp DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP).

MySQL Data Types	Internal Types
tinyint, smallint, mediumint, int, bigint,year	Long
float, double, decimal	Double
varchar, char, tinytext, text, mediumtext, longtext,set,json	String
date, datetime, timestamp, time	Date
bit, bool	Boolean
tinyblob, mediumblob, blob, longblob, varbinary	Bytes

tencent cloud

New User Offers

Next-Generation CDN：EdgeOne

Elasticsearch Service free trial

Free Tier

Tencent Cloud Startup Program

Special Offers

Lighthouse Special Offers

Cloud Object Storage Special Offers

Featured Products

New Products

Education

Tencent Cloud Online Education Solutions

Gaming

Gaming Solution

Game Media Solutions

E-commerce

E-commerce retail solutions

Audio & Video

Audio/Video Solution

LVB Recording Solution

Interactive Classroom Solution

Interactive Live Streaming Solution

Audio Chat Social Networking Solution

Financial Services

Financial Services Solution

Compute

Cloud Virtual Machine

Auto Scaling

Batch Compute

CVM Dedicated Host

Database

TencentDB for MySQL

TencentDB for Redis®

TencentDB for CTSDB

TDSQL for MySQL

Data Transfer Service

TencentDB for MongoDB

TencentDB for PostgreSQL

TencentDB for SQL Server

Video Service

Cloud Streaming Services

Video on Demand

Media Processing Service

Cloud Application Rendering

Cloud Contact Center

Game Multimedia Engine

Chat

Real-time Communication

Tencent Effect SDK

AI and Machine Learning

Image Creation Large Model

Face Fusion

eKYC

Optical Character Recognition

Video Creation Large Model

Industry Applications

Tencent HealthCare Omics Platform

Container and Middleware

TDMQ for CKafka

Serverless Cloud Function

Tencent Kubernetes Engine

Tencent Kubernetes Engine for Serverless

Networking

Cloud Load Balancer

Virtual Private Cloud

Direct Connect

Cloud Connect Network

NAT Gateway

VPN Connection

Bandwidth Package

Anycast Internet Acceleration

Elastic Network Interface

Flow Logs

Global Application Acceleration Platform

Security

Captcha

Cloud Workload Protection Platform

Data Security Governance Center

Key Management Service