tencent cloud

All product documents
TDMQ for CKafka
DocumentationTDMQ for CKafkaPractical TutorialConnector Practical TutorialQuerying Subscription to Database Change InfoAnalysis of Change Logs Tracked by MongoDB Change Streams
Analysis of Change Logs Tracked by MongoDB Change Streams
Download PDF
Last updated: 2024-01-09 14:56:36
Analysis of Change Logs Tracked by MongoDB Change Streams
Last updated: 2024-01-09 14:56:36
Download PDF

Overview

MongoDB uses change streams to track changes. However, to better search for change logs, you usually need to sync them to Elasticsearch or CLS.



This document uses connecting MongoDB to CKafka and distributing CKafka data to CLS as an example to describe how to use the data dump service of DataHub to analyze change logs tracked by change streams in MongoDB.

How It Works

For more information on data access configuration items for MongoDB, see Source Connector. You can set different configuration items to perform corresponding data processing tasks on the accessed change logs and then write them to a topic in the CKafka instance.

Prerequisites

TencentDB for MongoDB has been activated, or CLB is used to listen on the self-built MongoDB instance. The TCP:27017 port has been opened in the security group.
The CKafka service has been activated.
The CLS service has been activated.

Directions

Step 1. Create a data access task

1. Log in to the CKafka console.
2. Click Data Access on the left sidebar, select the region, and click Create Task.
3. In the pop-up window, select Asynchronously pulled data > MongoDB for Data Source Type.
4. Click Next and enter the task details.
5. Click Submit and wait for the task status to become Healthy.
6. When MongoDB data changes, you can see that there will be incremental messages in the selected topic in the CKafka instance.

Step 2. Create a data distribution task

1. Log in to the CKafka console.
2. Click Data Distribution on the left sidebar, select the region, and click Create Task.
3. Select Cloud Log Service (CLS) as the Target Type and click Next.
4. Enter the task details and select the same CKafka instance and topic as those used in the data access task, so that produced messages can be directly consumed.
5. Click Submit and wait for the task status to become Healthy.
Note:
When a task is in Healthy status and incremental messages are written to the topic, they will be directly consumed to the specified CLS log topic.

Step 3. View the distributed data

1. Log in to the CLS console.
2. Select Search and Analysis on the left sidebar, select the logset ID and log topic ID entered during distribution task creation, and you can view the change logs of MongoDB.
3. You can search by keyword to directly get the required logs.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon