tencent cloud

All product documents
TDMQ for CKafka
DocumentationTDMQ for CKafkaGeneral ReferencesConnectorWhat Is a Signaling Table
What Is a Signaling Table
Download PDF
Last updated: 2024-09-09 21:47:20
What Is a Signaling Table
Last updated: 2024-09-09 21:47:20
Download PDF

Background

The debezium connector initially synchronizes the existing data of the table only when a connection task is created. Subsequent newly added tables cannot trigger the synchronization of existing data. To support the synchronization of existing data for newly added tables, debezium uses a signal mode to notify the connector to trigger the synchronization of existing data for newly added tables.

Principles

You need to create a signaling table in the subscribed database. When you want to trigger the synchronization of existing data for a new table, insert the relevant information into the signaling table. Additionally, the connector needs to subscribe to this signaling table. When the connector receives a message from the signaling table, the synchronization of existing data for newly added tables will be triggered.

Notes

1. Since the subscription to the signaling table (dip_signal_taskId) messages has been added, the target Topic will include messages from the signaling table, which will require the business side to perform appropriate filtering.
2. Ensure that the user configured in Connection Management has the permissions to create, modify, and delete tables in the database. (This is only necessary for operations on the signaling table.)
3. If the signaling table is used to synchronize existing data for a newly added table, there may be instances of duplicate incremental data. To avoid duplicate data, you can pause the insert, update, and delete operations on the new table, edit the connector task's data source to add the table, and then resume the operations on the new table. Alternatively, you can perform idempotent processing downstream.
4. If you did not select the option to synchronize existing data when the task is created, and later select the option when you modified the data source to add a table with no change messages generated during this period, Debezium will synchronize the existing data of both the old and new tables (not related to the signaling table) by default. If there were incremental change messages in the old table before modifying the data source, then when you modify the data source to add a table and select the existing data synchronization, only the existing data of the new table will be synchronized (triggered by the signaling table).
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon