tencent cloud

All product documents
Cloud Log Service
DocumentationCloud Log ServiceOperation GuideLog CollectionCollection Overview
Collection Overview
Download PDF
Last updated: 2024-01-20 17:14:28
Collection Overview
Last updated: 2024-01-20 17:14:28
Download PDF

Overview

CLS provides log collection clients that allow you to collect your application logs and import them into CLS through APIs or SDKs. Currently, CLS requires logs to be uploaded in a structured manner as key-value pairs.

Log Structuring

The structuring of logs is to store your log data on the CLS platform in key-value format. Structured logs can be searched for, analyzed, and shipped based on specified keys. CLS allows you to report structured data directly. For more information, see the following:
For example, a local raw log is as follows:
10.20.20.10;[Tue Jan 22 14:49:45 CST 2019 +0800];GET /online/sample HTTP/1.1;127.0.0.1;200;647;35;http://127.0.0.1/
Specify that the log is parsed by separator and select semicolon (;) as the separator. Then the log can be parsed into multiple field groups and each group is organized in key-value pairs. A key name is defined for each key as follows:
IP: 10.20.20.10
time: [Tue Jan 22 14:49:45 CST 2019 +0800]
request: GET /online/sample HTTP/1.1
host: 127.0.0.1
status: 200
length: 647
bytes: 35
referer: http://127.0.0.1/
LogListener provides various parsing modes, in which reported logs with full text in a single line and full text in multi lines are both structured. LogListener adds __CONTENT__ as a key and uses the original text as the value by default. The key-value pair is as follows:
LogListener Parsing Mode
Key
Description
Full text in a single line/Full text in multi lines
__CONTENT__
__CONTENT__ is the key name by default.
JSON format
Name in the JSON file
The name and value in the original text of the JSON log are used as a key-value pair.
Separator format
Custom
After dividing fields using separators, you need to define a key name for each field group.
Full regular expression
Custom
After extracting fields based on a regular expression, you need to define a key name for each field group.

Collection Methods

CLS provides multiple methods for data collection:
Collection Method
Description
API
You can call CLS APIs to upload structured logs to CLS. For more information, see Uploading Log via API.
SDK
You can use SDKs to upload structured logs to CLS. For more information, see Collection via SDK.
LogListener client
LogListener is a log collection client provided by CLS. You can quickly access CLS by simply configuring LogListener in the console. For more information, see LogListener Use Process.
A comparison of the collection methods is as follows:
Category
Collection via LogListener
Collection via API
Code modification
Provides a non-intrusive collection method for applications, without code modification.
Reports logs only after modifying application code.
Resumable upload
Supports resumable upload of logs.
Automatically implemented by code.
Retransmission upon failure
Provides an inherent retry mechanism.
Automatically implemented by code.
Local cache
Supports local cache, ensuring data integrity during peak hours.
Automatically implemented by code.
Resource occupation
Occupies resources such as memory and CPU resources.
Occupies no additional resources.

Accessing Log Sources

You can access different log sources in different ways. For more information, see the following tables:
Log source environment
System Environment
Recommended Access Method
Linux/Unix
LogListener collection / Upload over Kafka / Log upload via API
Windows
iOS/Android/Web
Tencent Cloud service logs
For more information, see Tencent Cloud Service Log Access.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 available.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon