- Release Notes and Announcements
- User Guide
- Product Introduction
- Purchase Guide
- Getting Started
- User Guide
- Quota Management
- Managing Functions
- Web Function Management
- Log Management
- Concurrence Management
- Trigger Management
- A Custom Domain Name
- Version Management
- Alias Management
- Permission Management
- Managing Monitors and Alarms
- Network Configuration
- Layer Management
- Execution Configuration
- Extended Storage Management
- DNS Caching Configuration
- Resource Managed Mode Management
- Triggers
- Development Guide
- Developer Tools
- Code Development
- Web Framework Development
- Deploying Framework on Command Line
- Quickly Deploying Egg Framework
- Quickly Deploying Express Framework
- Quickly Deploying Flask Framework
- Quickly Deploying Koa Framework
- Quickly Deploying Laravel Framework
- Quickly Deploying Nest.js Framework
- Quickly Deploying Next.js Framework
- Quickly Deploying Nuxt.js Framework
- Quickly Deploying Django Framework
- Practical Tutorial
- Overview
- Solutions with Tencent Cloud Services
- Business Development
- ServerlessFramework Practices
- API Gateway
- TRTC Practices
- COS Practices
- CKafka Practice
- CLS
- CLB Practice
- MPS
- CDN
- CDWPG
- VOD
- SMS
- ES
- Scheduled Task
- Video Processing
- Success Stories
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Trigger APIs
- Function APIs
- CopyFunction
- Invoke
- UpdateFunctionConfiguration
- UpdateFunctionCode
- ListFunctions
- GetFunctionLogs
- GetFunction
- DeleteFunction
- CreateFunction
- PublishVersion
- ListVersionByFunction
- GetFunctionAddress
- DeleteAlias
- UpdateAlias
- ListAliases
- GetAlias
- CreateAlias
- PutTotalConcurrencyConfig
- PutReservedConcurrencyConfig
- PutProvisionedConcurrencyConfig
- GetReservedConcurrencyConfig
- GetProvisionedConcurrencyConfig
- DeleteReservedConcurrencyConfig
- DeleteProvisionedConcurrencyConfig
- UpdateFunctionEventInvokeConfig
- GetFunctionEventInvokeConfig
- InvokeFunction
- GetRequestStatus
- Namespace APIs
- Layer Management APIs
- Async Event Management APIs
- Other APIs
- Function and Layer Status Description
- Data Types
- Error Codes
- SDK Documentation
- FAQs
- Related Agreement
- Contact Us
- Glossary
- Release Notes and Announcements
- User Guide
- Product Introduction
- Purchase Guide
- Getting Started
- User Guide
- Quota Management
- Managing Functions
- Web Function Management
- Log Management
- Concurrence Management
- Trigger Management
- A Custom Domain Name
- Version Management
- Alias Management
- Permission Management
- Managing Monitors and Alarms
- Network Configuration
- Layer Management
- Execution Configuration
- Extended Storage Management
- DNS Caching Configuration
- Resource Managed Mode Management
- Triggers
- Development Guide
- Developer Tools
- Code Development
- Web Framework Development
- Deploying Framework on Command Line
- Quickly Deploying Egg Framework
- Quickly Deploying Express Framework
- Quickly Deploying Flask Framework
- Quickly Deploying Koa Framework
- Quickly Deploying Laravel Framework
- Quickly Deploying Nest.js Framework
- Quickly Deploying Next.js Framework
- Quickly Deploying Nuxt.js Framework
- Quickly Deploying Django Framework
- Practical Tutorial
- Overview
- Solutions with Tencent Cloud Services
- Business Development
- ServerlessFramework Practices
- API Gateway
- TRTC Practices
- COS Practices
- CKafka Practice
- CLS
- CLB Practice
- MPS
- CDN
- CDWPG
- VOD
- SMS
- ES
- Scheduled Task
- Video Processing
- Success Stories
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Trigger APIs
- Function APIs
- CopyFunction
- Invoke
- UpdateFunctionConfiguration
- UpdateFunctionCode
- ListFunctions
- GetFunctionLogs
- GetFunction
- DeleteFunction
- CreateFunction
- PublishVersion
- ListVersionByFunction
- GetFunctionAddress
- DeleteAlias
- UpdateAlias
- ListAliases
- GetAlias
- CreateAlias
- PutTotalConcurrencyConfig
- PutReservedConcurrencyConfig
- PutProvisionedConcurrencyConfig
- GetReservedConcurrencyConfig
- GetProvisionedConcurrencyConfig
- DeleteReservedConcurrencyConfig
- DeleteProvisionedConcurrencyConfig
- UpdateFunctionEventInvokeConfig
- GetFunctionEventInvokeConfig
- InvokeFunction
- GetRequestStatus
- Namespace APIs
- Layer Management APIs
- Async Event Management APIs
- Other APIs
- Function and Layer Status Description
- Data Types
- Error Codes
- SDK Documentation
- FAQs
- Related Agreement
- Contact Us
- Glossary
You can write an SCF function to process the logs collected in the CLS service. By passing the collected logs as a parameter, the function can be invoked and the function code can process and analyze the data or dump it to other Tencent Cloud services.
Characteristics of CLS triggers:
Push model
CLS monitors the specified log topic, aggregates data within a certain period of time, and invokes the associated function to push the event data to the function.
Async invocation
a CLS trigger always invokes a function asynchronously, and the result is not returned to the invoker. For more information on invocation types, please see "Invocation Types" in How It Works.
CLS Trigger Attributes
Logset: configure the CLS logset you want to connect to. It can only be a logset in the same region as the function.
Log topic: configure the CLS log topic you want to connect to, which is the smallest unit for managing and configuring CLS triggers.
Maximum waiting time: configure the longest waiting time for a single event pull.
CLS Trigger Consumption and Message Delivery
After consuming messages, the CLS backend consumption module will encapsulate them into event structures according to the max waiting time, delivery process, and message body size and then initiate async function invocation. The applicable limits are as follows:
Maximum waiting time
at present, the SCF backend consumption module requires this value to be between 3s and 300s to avoid an excessive latency before consumption. For example, if the maximum waiting time is set to 60s, the consumption template will aggregate log data once every 60s and deliver it to the function.
Event size limit for async invocation
128 KB. For more information, please see Limits. If a message of the log topic is large (for example, if the message body size exceeds 128 KB after being compressed by the backend component within the maximum waiting time), then due to the limit of 128 KB for async invocation, the system will extract and deliver the message body of 128 KB after being compressed by gzip in sequence from the event structure passed to the function instead of using the data aggregated within the maximum waiting time.
Delivery process
if a non-retriable error occurs during the delivery process, such as
AccessDeniedException or ResourceNotFoundException, the CLS trigger will pause the transfer retry logic.Note:
In the message delivery process, the time aggregation may vary by combination; that is, the number of messages in each event structure ranges from 1 to the maximum waiting time. If the configured maximum waiting time is too long, there may be cases where the aggregation time in an event structure will never reach the maximum aggregation time.
After the event content is obtained by the function, each message can be guaranteed for processing by loop handling, and it is not assumed that the message time passed each time is constant.
Event Message Structure for CLS Trigger
When the specified CLS trigger receives a message, the CLS backend consumption module will consume the message and encapsulate it to asynchronously invoke your function. In order to ensure the efficiency of data transfer in a single triggering action, the value of the data field is a Base64-encoded Gzip document.
{"clslogs": {"data": "ewogICAgIm1lc3NhZ2VUeXBlIjogIkRBVEFfTUVTU0FHRSIsCiAgICAib3duZXIiOiAiMTIzNDU2Nzg5MDEyIiwKICAgICJsb2dHcm91cCI6I..."}}
After being decoded and decompressed, the log data will look like the following JSON body (using decoded CLS Logs message data as an example):
{"topic_id": "xxxx-xx-xx-xx-yyyyyyyy","topic_name": "testname","records": [{"timestamp": "1605578090000000","content": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}, {"timestamp": "1605578090000000","content": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}]}
The data structures are as detailed below:
Structure | Description |
topic_id | Log topic ID |
topic_name | Log topic name |
timestamp | Log production time (timestamp at the microsecond level) |
content | Log content |
Yes
No
Was this page helpful?