tencent cloud

All product documents
Cloud Log Service
DocumentationCloud Log ServiceAPI DocumentationLog APIsUploadLog
UploadLog
Download PDF
Last updated: 2024-04-15 19:58:32
UploadLog
Last updated: 2024-04-15 19:58:32
Download PDF

1. API Description

Domain name for API request: cls.tencentcloudapi.com.

Note

To ensure log data reliability and help you use CLS more efficiently, we recommend you use the optimized API to upload logs. For more information about the API, see Uploading Log via API.

For the optimized API, we have developed an SDK (available in multiple languages) that provides features including async sending, resource control, automatic retry, graceful shutdown, and detection-based reporting. For details, see Uploading Log via SDK.

UploadLog allows you to synchronously upload log data. If you still want to continue to use this API instead of the optimized one, read this document.

Feature Description

This API is used to write logs to a specified log topic.

CLS provides the following two modes:

Load balancing mode

In this mode, logs will be automatically written to a target partition among all readable/writable partitions under the current log topic based on the load balancing principle. This mode is suitable for scenarios where sequential consumption is not needed.

Hash routing mode

In this mode, data will be written to a target partition that meets the range requirements based on the carried hash value (X-CLS-HashKey). For example, a log source can be bound to a topic partition through HashKey, strictly guaranteeing the sequence of the data written to and consumed in this partition.

Input parameters (pb binary streams in body)

Parameter Type Location Required Description
logGroupList message pb Yes The logGroup list, which describes the encapsulated log groups. We recommend you enter up to five logGroup values.

LogGroup description:

Parameter Required Description
logs Yes Log array consisting of multiple Log values. The Log indicates a log, and a LogGroup can contain up to 10,000 Log values.
contextFlow No Unique LogGroup ID, which should be passed in if the context feature needs to be used. Format: "{Context ID}-{LogGroupID}".
Context ID: Uniquely identifies the context (a series of log files that are continuously scrolling or a series of logs that need to be sequenced), which is a 64-bit integer hex string.
LogGroupID: A 64-bit integer hex string that continuously increases, such as 102700A66102516A-59F59.
filename No Log filename
source No Log source, which is generally the machine IP
logTags No List of log tags

Log description:

Parameter Required Description
time Yes Unix timestamp of log time in seconds or milliseconds (recommended)
contents No Log content in key-value format. A log can contain multiple key-value pairs.

Content description:

Parameter Required Description
key Yes Key of a field group in one log, which cannot start with _.
value Yes Value of a field group. The value of one log cannot exceed 1 MB and the total value in LogGroup cannot exceed 5 MB.

LogTag description:

Parameter Required Description
key Yes Key of a custom tag
value Yes Value corresponding to the custom tag key

pb Compilation Example

This example shows you how to use the protoc compiler to compile a pb description file into a log upload API in C++.

Note：

Currently, protoc supports compilation in multiple programming languages such as Java, C++, and Python. For more information, see protoc.

1. Install protocol buffers

Download Protocol Buffers, decompress the package, and install the tool. The version used in the example is protobuf 2.6.1 running on CentOS 7.3. Run the following command to decompress the protobuf-2.6.1.tar.gz package to the /usr/local directory and go to the directory:

tar -zxvf protobuf-2.6.1.tar.gz -C /usr/local/ && cd /usr/local/protobuf-2.6.1

Run the following commands to start compilation and installation and configure the environment variables:

[root@VM_0_8_centos protobuf-2.6.1]# ./configure 
[root@VM_0_8_centos protobuf-2.6.1]# make && make install
[root@VM_0_8_centos protobuf-2.6.1]# export PATH=$PATH:/usr/local/protobuf-2.6.1/bin

After the compilation succeeds, run the following command to check the version:

[root@VM_0_8_centos protobuf-2.6.1]# protoc --version
liprotoc 2.6.1

2. Create a pb description file

A pb description file is an agreed-on data interchange format for communication. To upload logs, compile the specified protocol format to an API in the target programming language and add the API to the project code. For more information, see protoc.

Create a pb message description file cls.proto based on the pb data format content specified by CLS.

Note：

The pb description file content cannot be modified, and the filename must end with .proto.

The content of cls.proto (pb description file) is as follows:

package cls;

message Log
{
    message Content
    {
        required string key   = 1; // Key of each field group
        required string value = 2; // Value of each field group
    }
    required int64   time     = 1; // Unix timestamp
    repeated Content contents = 2; // Multiple key-value pairs in one log
}

message LogTag
{
    required string key       = 1;
    required string value     = 2;
}

message LogGroup
{
    repeated Log    logs        = 1; // Log array consisting of multiple logs
    optional string contextFlow = 2; // This parameter does not take effect currently
    optional string filename    = 3; // Log filename
    optional string source      = 4; // Log source, which is generally the machine IP
    repeated LogTag logTags     = 5;
}

message LogGroupList
{
    repeated LogGroup logGroupList = 1; // Log group list
}

3. Compile and generate the API

This example uses the proto compiler to generate a C++ file in the same directory as the cls.proto file. Run the following compilation command:

protoc --cpp_out=./ ./cls.proto

Note：

--cpp_out=./ indicates that the file will be compiled in cpp format and output to the current directory. ./cls.proto indicates the cls.proto description file in the current directory.

After the compilation succeeds, the code file in the corresponding programming language will be generated. This example generates the cls.pb.h header file and cls.pb.cc code implementation file as shown below:

[root@VM_0_8_centos protobuf-2.6.1]# protoc --cpp_out=./ ./cls.proto
[root@VM_0_8_centos protobuf-2.6.1]# ls
cls.pb.cc cls.pb.h cls.proto

4. Call the API

Import the generated cls.pb.h header file into the code and call the API for data encapsulation.

A maximum of 100000 requests can be initiated per second for this API.

Note: this API can only use HTTP method POST with application/octet-stream content type.

2. Input Parameters

The following request parameter list only provides API request parameters and some common parameters. For the complete common parameter list, see Common Request Parameters.
Note: this API only supports the application/octet-stream type, and can only be called using the signature method V3, and the request parameters must be placed in the request header.

Parameter Name Required Type Description
X-TC-Action Yes String Common Params. The value used for this API: UploadLog.
X-TC-Version Yes String Common Params. The value used for this API: 2020-10-16.
X-TC-Region No String Common Params. This parameter is not required for this API.
X-CLS-TopicId Yes String Topic ID
X-CLS-HashKey No String Topic partition where data will be written into by HashKey
X-CLS-CompressType No String Compression type

3. Output Parameters

Parameter Name Type Description
RequestId String The unique request ID, generated by the server, will be returned for every request (if the request fails to reach the server for other reasons, the request will not obtain a RequestId). RequestId is required for locating a problem.

4. Example

Example1 Uploading Logs

This example shows you how to upload logs.

Input Example

POST / HTTP/1.1
Host: cls.tencentcloudapi.com
Content-Type: application/octet-stream
X-TC-Action: UploadLog
<Common request parameters>

{
    "HashKey": "abc",
    "TopicId": "abc",
    "CompressType": "abc"
}

Output Example

{
    "Response": {
        "RequestId": "6ef60bec-0242-43af-bb20-270359fb54a7"
    }
}

5. Developer Resources

SDK

TencentCloud API 3.0 integrates SDKs that support various programming languages to make it easier for you to call APIs.

Command Line Interface

6. Error Code

The following only lists the error codes related to the API business logic. For other error codes, see Common Error Codes.

Error Code Description
FailedOperation Operation failed.
FailedOperation.MissingContent Invalid Content.
FailedOperation.ReadQpsLimit The read QPS exceeds the limit.
FailedOperation.TopicClosed The log topic has been disabled.
FailedOperation.TopicIsolated The log topic has been isolated.
FailedOperation.WriteQpsLimit The write QPS exceeds the limit.
FailedOperation.WriteTrafficLimit The write traffic exceeds the limit.
InternalError Internal error.
InvalidParameter Incorrect parameter.
InvalidParameter.Content Invalid Content.
LimitExceeded.LogSize The log size exceeds the limit.
LimitExceeded.Tag The number of tags exceeds the limit.
MissingParameter Missing parameter.
OperationDenied Operation denied.
OperationDenied.ACLFailed ACL verification failed.
OperationDenied.AccountDestroy The account has been terminated.
OperationDenied.AccountIsolate The account has overdue payments.
OperationDenied.AccountNotExists The account does not exist.
ResourceNotFound.PartitionNotExist The partition does not exist.
ResourceNotFound.TopicNotExist The log topic does not exist.
UnsupportedOperation Unsupported operation.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon