tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Cloud Log Service
    Documentation

    ETL Log Processing

    Last updated: 2024-01-20 17:55:44

      Overview

      This document describes how to use SCF to process CLS logs. CLS is mainly used for log collection, while SCF mainly provides node computing capabilities. The data flow is as follows:
      
      

      Prerequisites

      You have logged in to the CLS console.

      Directions

      Creating log topic

      Create two log topics as instructed in Managing Log Topic.
      Note:
      As both the source and destination of data ETL are CLS, you need to create at least two topics.

      Creating SCF function

      Create a function as instructed in Creating and Testing Function. The main parameters are as follows:
      Region: Select the Beijing region.
      Function Name: Enter "CLSdemo".
      Creation Method: Click Template and select the CLSLogETL template.
      Note:
      You should select the VPC and subnet of CLS for the created function on the Function Configuration page.

      Configuring CLS trigger

      1. Log in to the CLS console.
      2. On the left sidebar, click Log Topic to enter the log topic management page.
      3. Find the log topic you just created and click its ID/name to enter the log topic details page.
      4. On the log topic details page, select the Function Processing tab and click Create.
      5. In the Function Processing pop-up window, add the created function and click OK. The main parameter information is as follows. Use the default values for the remaining configuration items.
      Namespace: Select the function namespace.
      Function Name: Select the function created in the Creating SCF function step.
      Alias: Select a function alias.
      Batch Window: Configure the longest waiting time for a single event pull. Default value: 60s.

      Testing function

      1. Download the log file in the test sample, extract demo-scf1.txt, and import it to the source CLS service.
      2. Switch to the SCF console to view the execution result. On the function details page, select the Log Query tab to view the printed log information.
      3. Switch to the target CLS service to view the data processing result.
      Note:
      You can write specific data processing methods as needed.
      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy