You will get an error message in the console if you fail to create a dump task.
Generally, you can troubleshoot the problem based on the error message:
1. If the error message says "Connection to XXX failed. Check whether the username and password are correct", you can check whether the entered username and password are correct, and if so, recreate the task with correct parameters.
2. If the error message says "Connection failed. Check whether the port is opened", you can check whether the port is opened. You can open the port in the security group as described in Analysis of Change Logs Tracked by MongoDB Change Streams.
3. If the error message says "The table xxx does not exist in the database", you can check whether the table exists in the database.
4. If the error message says "createLink fail", you can check whether the network connection fails, and contact us for assistance.
If the error message is unclear for you to troubleshoot, contact us for assistance.
What should I do if the dump task status is abnormal?
You will get an error message in the console if an exception occurs in the dump task.
Generally, you can troubleshoot the problem based on the error message:
1. If the error message says "The source CKafka instance does not exist" or "The target CKafka instance does not exist", you can check whether the instance is abnormal or has been deleted.
2. If the error message says "the source topic does not exist" or "The target topic does not exist", you can check whether the topic in the CKafka instance is deleted.
3. If the error message says "Check whether the account and password are correct", you can check whether the username and password have been changed, and if so, you need to update them in the task.
If the error message is unclear for you to troubleshoot, contact us for assistance.
What is task concurrency?
Concurrency is a concept that exists in all non-data reporting tasks, which is set to 1 by default when a task is created. During task execution, the system automatically detects the data heap.
1. If data heap occurs, the system will automatically expand the concurrency to increase the data throughput.
2. If the amount of data is low, the system will shrink the concurrency to avoid unnecessary resource use.
In data processing and data dump scenarios, the concurrency ranges from 1 to the number of partitions in the CKafka topic.
In the data integration scenario, the concurrency will be adjusted based on the upstream engine in the range of 1 to infinite as needed.
How does MongoDB data integration work?
MongoDB uses change streams to record data changes. When any data in the database changes, the application can get the changed data through the change stream mechanism. You can consider it as a trigger executed in the application. As for what data the application wants to get in what form, the data can be filtered and transformed through the aggregation framework.
The underlying dependency is the MongoDB's official Kafka source connector. The connector always listens on data changes in the database and stores the changed data in the Kafka topic.
Is MongoDB data partitionally sequential?
By default, MongoDB data is written to partitions by MongoDB object ID (as the key). Therefore, if the partitions are unchanged, the data can be partitionally sequential. If new partitions are added during subscription, rehashing will occur, and the changed data of the same data record during consumption will become non-sequential for a short time.
What should I do if a role is missing?
1. Find Access Management in the account list.
2. Select Roles on the left sidebar and click Create Role.
3. Select Tencent Cloud Product Service as the Role Entity.
4. Find Message Service (CKafka) in Enter Role Entity Info.
5. In Configure Role Policy, select policies such as QcloudCLSFullAccess and QcloudCOSFullAccess based on the dump target service and click Next.
6. Enter the role name DIP_QcsRole on the Review page and set the service role CKafka Connector as the description. This role will access your Tencent Cloud service resources within the permission scope of the associated policy.
Yes
No
Was this page helpful?