tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Documentation

CreateDataTransform

Last updated: 2024-04-15 19:58:39

    1. API Description

    Domain name for API request: cls.tencentcloudapi.com.

    This API is used to create a data processing task.

    A maximum of 20 requests can be initiated per second for this API.

    We recommend you to use API Explorer
    Try it
    API Explorer provides a range of capabilities, including online call, signature authentication, SDK code generation, and API quick search. It enables you to view the request, response, and auto-generated examples.

    2. Input Parameters

    The following request parameter list only provides API request parameters and some common parameters. For the complete common parameter list, see Common Request Parameters.

    Parameter Name Required Type Description
    Action Yes String Common Params. The value used for this API: CreateDataTransform.
    Version Yes String Common Params. The value used for this API: 2020-10-16.
    Region No String Common Params. This parameter is not required for this API.
    FuncType Yes Integer Task type. 1: Specify topic; 2: Dynamic creation. For details, please refer to Creating Processing Task Document (https://www.tencentcloud.com/document/product/614/63940?from_cn_redirect=1).
    SrcTopicId Yes String Source log topic
    Name Yes String Data processing task name
    EtlContent Yes String Data processing statement
    TaskType Yes Integer Processing type.
    1: Process preview using random data from the source log topic; 2: Process preview using user-defined test data; 3: Create real processing tasks.
    DstResources.N No Array of DataTransformResouceInfo Destination topic_id and alias of processing task. This parameter is required when FuncType=1, and not required when FuncType=2.
    EnableFlag No Integer Task status. Valid values: 1 (enabled) and 2 (disabled).
    PreviewLogStatistics.N No Array of PreviewLogStatistic Test data used for previewing the processing result

    3. Output Parameters

    Parameter Name Type Description
    TaskId String Task ID
    RequestId String The unique request ID, generated by the server, will be returned for every request (if the request fails to reach the server for other reasons, the request will not obtain a RequestId). RequestId is required for locating a problem.

    4. Example

    Example1 Creating a Data Processing Task

    This example shows you how to create a data processing task. Similar to the open source component Logstash, data processing provides capabilities such as filtering, cleaning, anonymization, enrichment, and distribution of log data. Domain Specific Language (DSL) functions are implemented based on Flink for data processing, allowing you to easily process log stream data.

    Input Example

    POST / HTTP/1.1
Host: cls.tencentcloudapi.com
Content-Type: application/json
X-TC-Action: CreateDataTransform
<Common request parameters>

{
    "EnableFlag": 1,
    "Name": "Data processing task",
    "FuncType": 0,
    "PreviewLogStatistics": [
        {
            "LineNum": 0,
            "FailReason": "errorcode",
            "DstTopicId": "81XXXXe5-e39e-4a1e-b2d4-a778df97d825",
            "LogContent": "XXXX",
            "Time": "2017-08-08 12:12:12"
        }
    ],
    "SrcTopicId": "xxxx-xx-xx-xx-xxxxxxxx",
    "TaskType": 0,
    "DstResources": [
        {
            "TopicId": "3d9bXXXX-05a4-4435-ac65-f43e684329b3",
            "Alias": "Alias"
        }
    ],
    "EtlContent": "xx-xx"
}

    Output Example

    {
    "Response": {
        "TaskId": "xxxx-xx-xx-xx-xxxxxxxx",
        "RequestId": "6ef60bec-0242-43af-bb20-270359fb54a7"
    }
}

    5. Developer Resources

    SDK

    TencentCloud API 3.0 integrates SDKs that support various programming languages to make it easier for you to call APIs.

    Command Line Interface

    6. Error Code

    The following only lists the error codes related to the API business logic. For other error codes, see Common Error Codes.

    Error Code Description
    FailedOperation Operation failed.
    InternalError Internal error.
    InvalidParameter Incorrect parameter.
    InvalidParameter.DataFromTaskConflict There is a data processing task conflict.
    InvalidParameter.InvalidEtlContent The data processing statement is invalid.
    MissingParameter Missing parameter.
    OperationDenied Operation denied.
    OperationDenied.ACLFailed ACL verification failed.
    OperationDenied.AccountDestroy The account has been terminated.
    OperationDenied.AccountIsolate The account has overdue payments.
    OperationDenied.AccountNotExists The account does not exist.
    ResourceNotFound The resource does not exist.
    ResourceNotFound.TopicNotExist The log topic does not exist.
    Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
    Privacy PolicyLegalCookie Policy