Shipping and Consumption

Recent Pages

Shipping and Consumption

Last updated: 2024-12-17 17:57:09

The restrictions are shown in the table below:
Type
Restriction Item
Description
Ship to COS
Number of Tasks Submitted
For each log topic, a maximum of 10 shipping to COS tasks can be configured.
﻿
Deliverable Data Scope
Data (including historical and real-time data) of log topics in the lifecycle.
﻿
Task Submission Delay
The latency of shipping to COS is about 60 seconds.
Note:
If you set the shipping interval to be 5 minutes, logs from 12:00 to 12:05, meeting the interval of 5 minutes, will be shipped as a whole to COS in a batch. With a latency of 60 seconds, this batch of logs will arrive at COS around 12:06.
﻿
Trigger Delivery
One shipping will be triggered when a partition meets any of the following conditions:
Size of logs to be shipped (accumulated log data). Range: 5 to 256 MB (uncompressed).
Shipping interval. Range: 5 to 15 minutes.
﻿
Cross-account/Cross-region
Cross-account: supported
Cross-region: not supported
Ship to CKafka
Number of Tasks Submitted
For each log topic, one shipping to CKafka task can be configured.
﻿
Deliverable Data Scope
At present, only real-time log data can be shipped, and historical data is not supported.
﻿
Task Submission Delay
The latency of shipping 99.9% of logs to the CKafka are within 5 seconds. (The target CKafka functions properly, has a throughput capacity sufficient to process messages from the CLS and the disk is not full or throttled). 
Note:
If the timestamp (value of _TIMESTAMP_) of a log collected to CLS is 12:00:00, the log will arrive at CKafka before 12:00:05.
﻿
Cross-account/Cross-region
Cross-account: supported
Cross-region: not supported
Consumption over Kafka
Versions
Kafka protocol version supported: 1.1.1 and earlier.
﻿
Number of consumer groups
Each log topic can be configured with 20 consumer groups.
﻿
Authentication Method
Only SASL_PLAINTEXT is supported.
﻿
Consumable data range
Once this feature is enabled, consumption will start from the latest data. Data will be retained for 2 hours. If data is not consumed within 2 hours, it will not be saved. 
Note:
For example, if you enable consumption over Kafka protocol at 12:00, you will consume data from 12:00. Until 15:00, you can consume data from 13:00 to 15:00. Data before 13:00 cannot be consumed.
﻿
Consumption latency
The consumption latency of 99.9% log data is within 5 seconds.
Note:
If the timestamp (value of _TIMESTAMP_) of a log collected to CLS is 12:00:00, the log will arrive at the consumer before 12:00:05.
﻿
Cross-account/Cross-region
Cross-account: Account A can be a collaborator of account B to consume the log topics of account B. Permissions need to be set for the collaborator. For details, see the permission configuration.
Cross-region: Cross-region consumption can be implemented through public network.

Contact Us

Contact our sales team or business advisors to help your business.

Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

7x24 Phone Support

tencent cloud

Recent Pages

Shipping and Consumption

Was this page helpful?

Was this page helpful?

Type	Restriction Item	Description
Ship to COS	Number of Tasks Submitted	For each log topic, a maximum of 10 shipping to COS tasks can be configured.
		Deliverable Data Scope	Data (including historical and real-time data) of log topics in the lifecycle.
		Task Submission Delay	The latency of shipping to COS is about 60 seconds. Note: If you set the shipping interval to be 5 minutes, logs from 12:00 to 12:05, meeting the interval of 5 minutes, will be shipped as a whole to COS in a batch. With a latency of 60 seconds, this batch of logs will arrive at COS around 12:06.
		Trigger Delivery	One shipping will be triggered when a partition meets any of the following conditions: Size of logs to be shipped (accumulated log data). Range: 5 to 256 MB (uncompressed). Shipping interval. Range: 5 to 15 minutes.
		Cross-account/Cross-region	Cross-account: supported Cross-region: not supported
Ship to CKafka	Number of Tasks Submitted	For each log topic, one shipping to CKafka task can be configured.
		Deliverable Data Scope	At present, only real-time log data can be shipped, and historical data is not supported.
		Task Submission Delay	The latency of shipping 99.9% of logs to the CKafka are within 5 seconds. (The target CKafka functions properly, has a throughput capacity sufficient to process messages from the CLS and the disk is not full or throttled). Note: If the timestamp (value of _TIMESTAMP_) of a log collected to CLS is 12:00:00, the log will arrive at CKafka before 12:00:05.
		Cross-account/Cross-region	Cross-account: supported Cross-region: not supported
Consumption over Kafka	Versions	Kafka protocol version supported: 1.1.1 and earlier.
		Number of consumer groups	Each log topic can be configured with 20 consumer groups.
		Authentication Method	Only SASL_PLAINTEXT is supported.
		Consumable data range	Once this feature is enabled, consumption will start from the latest data. Data will be retained for 2 hours. If data is not consumed within 2 hours, it will not be saved. Note: For example, if you enable consumption over Kafka protocol at 12:00, you will consume data from 12:00. Until 15:00, you can consume data from 13:00 to 15:00. Data before 13:00 cannot be consumed.
		Consumption latency	The consumption latency of 99.9% log data is within 5 seconds. Note: If the timestamp (value of _TIMESTAMP_) of a log collected to CLS is 12:00:00, the log will arrive at the consumer before 12:00:05.
		Cross-account/Cross-region	Cross-account: Account A can be a collaborator of account B to consume the log topics of account B. Permissions need to be set for the collaborator. For details, see the permission configuration. Cross-region: Cross-region consumption can be implemented through public network.

tencent cloud

Sign Up

Log in

Recent Pages

Shipping and Consumption

Was this page helpful?

Was this page helpful?