tencent cloud

All product documents
Serverless Cloud Function
DocumentationServerless Cloud FunctionPractical TutorialCLSDumping Logs to ES with SCF + CLS
Dumping Logs to ES with SCF + CLS
Download PDF
Last updated: 2024-12-02 16:35:34
Dumping Logs to ES with SCF + CLS
Last updated: 2024-12-02 16:35:34
Download PDF

Scenario

This document describes how to use SCF to dump CLS logs to ES. CLS is mainly used for log collection, and SCF mainly provides node computing capabilities for data processing. For the data processing flowchart, see Function Processing Overview.

Directions

Creating logset and topic

1. Log in to the CLS console, and click Log Topic on the left sidebar.
2. On the logset management page, select the region of the logset at the top.
3. Click Create Log Topic and enter relevant information in the Create Logset pop-up window:
Log Topic Name: Enter project_test for example.
Logset Name: Enter nginx for example.

4. Click OK.
5. The log topic is successfully added, and you will be redirected to the log topic management page.


Creating an SCF function

1. Log in to the SCF console and enter the Functions page.
2. At the top of the Functions page, select the Beijing region and click Create to go to the function creating page and configure the following parameters:
Creation method: Select Template.
Fuzzy search: Enter CLSToElasticsearch and search.
3. You can click Learn more in a template to view its details or download the template. Click Next to enter the Function configuration page.



4. In the Basic configurations section, the function name has been automatically generated and can be modified as needed. Follow the prompts to configure environment variables and execution role.
Environment variable: Add the following environment variables and configure them.


key
value
Required/Optional
ES_Address
ES service address
Yes
ES_User
ES username, which is elastic by default.
Yes
ES_Password
ES password.
Yes
Execution Role: Select Enable and click Configure and use SCF template role.
5. In Network configuration, select Enable for VPC, and select the same VPC and subnet as Elasticsearch.

6. Keep the default configuration and click Complete to complete the function creation.

Configuring CLS triggers

1. Log in to the CLS console and click Logset on the left sidebar.
2. Locate the existing logset you want, and click View under Operation on the right to go to the logset details page.
3. On the log topic details page, select Function Processing and click Create. Add the created function in the Function Processing pop-up window.

The main parameter information is as follows. Use the default values for the remaining configuration items.
Namespace: Select the function namespace.
Function name: Select the function created in the Creating SCF function step.
Alias: Select a function alias.
Max waiting time: Configure the longest waiting time for a single event pull. Default value: 60s.

Testing the function

1. Download the log file in the test sample, extract demo-scf1.txt, and import it to the source CLS service.
2. Go to the SCF console to view the execution result. Select the Log Query tab on the function details page to view the printed log information.

3. Log in to the ES console to view the data dumping and processing result.
Note:
You can write specific data processing methods as needed.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon