tencent cloud

All product documents
Stream Compute Service
DocumentationStream Compute ServiceETL Developer GuideConnectorsMySQL Sources
MySQL Sources
Download PDF
Last updated: 2023-11-08 16:21:31
MySQL Sources
Last updated: 2023-11-08 16:21:31
Download PDF

Overview

A MySQL data source can read all or new data from the MySQL database and supports exactly-once semantics. It uses Debezium for change data capture (CDC). ‌‍It works as follows:
1. Acquires a global read lock to prohibit write operations by other database clients.
2. Starts a repeatable read transaction to ensure that data is always read from the same checkpoint.
3. Reads the current binlog location.
4. ‍Reads the schema of the database and table.
5. Releases the global read lock to allow write operations by other database clients.
6. Scans the table and, after all data is read, obtains the changes made after the binlog location in step 3.
Records the binlog location as checkpoints are created regularly during the execution of the job, so that in case of a crash, processing can resume from the last recorded binlog location. This ensures exactly-once semantics.

Type mapping

MySQL Field Type
Flink Field Type
TINYINT
TINYINT
SMALLINT
SMALLINT
TINYINT UNSIGNED
INT
INT
MEDIUMINT
SMALLINT UNSIGNED
BIGINT
BIGINT
INT UNSIGNED
BIGINT UNSIGNED
DECIMAL(20, 0)
FLOAT
FLOAT
DOUBLE
DOUBLE
DOUBLE PRECISION
NUMERIC(p, s)
DECIMAL(p, s)
DECIMAL(p, s)
BOOLEAN
BOOLEAN
TINYINT(1)
DATE
DATE
TIME [(p)]
TIME [(p)] [WITHOUT TIMEZONE]
DATETIME [(p)]
TIMESTAMP [(p)] [WITHOUT TIMEZONE]
TIMESTAMP [(p)]
TIMESTAMP [(p)]
TIMESTAMP [(p)] WITH LOCAL TIME ZONE
CHAR(n)
STRING
VARCHAR(n)
TEXT
BINARY
BYTES
VARBINARY
BLOB

Notes

User permissions

The user of the source database must have the following permissions: SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT, SELECT, and RELOAD.

Database parameters

‍The running value of binlog_row_image should be FULL.

WITH parameters

MySQL data sources are developed based on MySQL CDC. ‍The two have identical WITH parameters. For details on how to configure them, see MySQL CDC.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon