DataHub supports accessing different types of data generated by various data sources for unified management and distribution to downstream offline/online processing systems, forming a clear data flow channel.
This document takes HTTP data as an example to describe how to create an active data reporting task and modify the task configuration in the CKafka console.
Directions
Creating data access task
Prerequisites: You have created a CKafka instance and a topic.
2. Click Data Access on the left sidebar, select the region, and click Create Task.
3. In the pop-up window, select Actively reported data for Data Source Type.
4. Click Next, enter the task name, and select the created CKafka instance and topic.
Task Name: Enter the task name. It can only contain letters, digits, underscores, or symbols ("-" and ".").
CKafka Instance: Select the target CKafka instance.
Target Topic: Select the target CKafka topic for data shipping. The data distribution feature cannot be normally used if ACL policies are configured for the selected topic.
Schema: After a schema is associated, it will be used to verify data format. If there is no appropriate schema, you can click Create Schema to enter the schema creation page.
QPS Limit: Enter the QPS limit.
5. Click Submit. After the task is created successfully, access point information will be generated.
6. Copy the access point information to the SDK to write data.
2. Click Data Access on the left sidebar and click the ID of the target task to enter its basic information page. In the basic information module, you can associate/disassociate schemas.
2. Click Data Access on the left sidebar and click the ID of the target task to enter its basic information page.
3. Select the Monitoring tab to view the monitoring data of the target topic.
Pausing task
On the Data Access page, click Pause in the Operation column of the target task to pause the task.
Note:
If you find that the data access task affects the normal use of CKafka, you can pause it.
Resuming task
On the Data Access page, click Resume in the Operation column of the target task to resume the paused task.
Note:
A paused task can be resumed to continue dumping data.
Deleting task
On the Data Access page, click Delete in the Operation column of the target task and click OK in the pop-up window to delete the task.
Note:
Once the task is deleted, data access will be stopped and the task record will be deleted, but the previously dumped data and CKafka instance involved will not be affected.
A task cannot be recovered once deleted. Proceed with caution.
Yes
No
Was this page helpful?