tencent cloud

All product documents
Data Transfer Service
DocumentationData Transfer ServiceData SyncSynchronize to KafkaMySQL/MariaDB/Percona/TDSQL-C MySQL Sync to KafkaDemo DescriptionJSON Demo Description
JSON Demo Description
Download PDF
Last updated: 2024-07-08 19:02:56
JSON Demo Description
Last updated: 2024-07-08 19:02:56
Download PDF
The demo for each programming language uses JSON for serialization and contains a Record definition file. In the demo for Java, the path of the definition file is consumerDemo-json-java\src\main\java\json\FlatRecord.java.

‍Type of Field in Record

Field Name in Record
Description
id
The globally incremental ID.
version
The protocol version, which is v1 currently.
messageType
The message type. Enumerated values: INSERT, UPDATE, DELETE, DDL, BEGIN, COMMIT, HEARTBEAT, CHECKPOINT.
fileName
The name of the binlog file where the current record is located.
position
The end offset of the current record in the binlog in the format of End_log_pos@binlog file number. For example, if the current record is in file mysql-bin.000004 and the end offset is 2196, then the value of this parameter will be 2196@4.
safePosition
The start offset of the current transaction in the binlog, which is in the same format as described above.
timestamp
The time when the data was written to the binlog, which is a UNIX timestamp in seconds.
gtid
The current GTID, such as c7c98333-6006-11ed-bfc9-b8cef6e1a231:9.
transactionId
The transaction ID, which is generated only for COMMIT events.
serverId
The server ID of the source database, which can be viewed by running SHOW VARIABLES LIKE 'server_id'.
threadId
The ID of the session that committed the current transaction, which can be viewed by running SHOW processlist;.
sourceType
The source database type, which currently can only be MySQL.
sourceVersion
The source database version, which can be viewed by running: select version();.
schemaName
Database name.
tableName
Table name.
objectName
Format: Database name.table name.
columns
The definitions of columns in the table.
oldColumns
The data of the row before DML execution. If the message is an INSERT message, the array will be null.
newColumns
The data of the row after DML execution. If the message is a DELETE message, the array will be null.
sql
The DDL SQL statement.
executionTime
The DDL execution duration in seconds.
heartbeatTimestamp
The timestamp of the heartbeat message in seconds. This field is present only for heartbeat messages.
syncedGtid
The collection of GTIDs parsed by DTS in the format of c7c98333-6006-11ed-bfc9-b8cef6e1a231:1-13.
fakeGtid
Whether the current GTID is forged. If gtid_mode is not enabled, DTS will forge a GTID.
pkNames
If the table in the source database has a primary key, this parameter will be carried in the DML message; otherwise, it will not be carried.
readerTimestamp
The time when DTS processed the current data record, which is a UNIX timestamp in milliseconds.
tags
The status_vars in QueryEvent. For more information, see QueryEvent.
total
The total number of message segments if the message is segmented. This field is invalid on the current version (version=1) and is reserved for extension.
index
The index of a message segment if the message is segmented. This field is invalid on the current version (version=1) and is reserved for extension.

MySQL Column Attribute in Record

name: The column name.
dataTypeNumber: The type of the data recorded in the binlog. For values, see MySQL source code documentation.
isKey: Whether the current key is the primary key.
originalType: The type defined in DDL.

MySQL Data Type Conversion Logic

In the JSON protocol, all MySQL data types are converted to strings.
String types such as varchar are all converted to UTF-8 encoding.
Numeric types are all converted to strings equal to the value, such as "3.0".
Time types are output in the format of yyyy-dd-mm hh:MM:ss.milli.
Timestamp types are output as the number of milliseconds.
Binary types such as binary and blob are output as strings equal to their hex values, such as "0xfff".
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon