tencent cloud

All product documents
APIs
DocumentationAPIsCloud Log ServiceData Processing APIsCreateDataTransform
CreateDataTransform
Last updated: 2024-04-15 19:58:39
CreateDataTransform
Last updated: 2024-04-15 19:58:39

1. API Description

Domain name for API request: cls.tencentcloudapi.com.

This API is used to create a data processing task.

A maximum of 20 requests can be initiated per second for this API.

We recommend you to use API Explorer
Try it
API Explorer provides a range of capabilities, including online call, signature authentication, SDK code generation, and API quick search. It enables you to view the request, response, and auto-generated examples.

2. Input Parameters

The following request parameter list only provides API request parameters and some common parameters. For the complete common parameter list, see Common Request Parameters.

Parameter Name Required Type Description
Action Yes String Common Params. The value used for this API: CreateDataTransform.
Version Yes String Common Params. The value used for this API: 2020-10-16.
Region No String Common Params. This parameter is not required for this API.
FuncType Yes Integer Task type. 1: Specify topic; 2: Dynamic creation. For details, please refer to Creating Processing Task Document (https://www.tencentcloud.com/document/product/614/63940?from_cn_redirect=1).
SrcTopicId Yes String Source log topic
Name Yes String Data processing task name
EtlContent Yes String Data processing statement
TaskType Yes Integer Processing type.
1: Process preview using random data from the source log topic; 2: Process preview using user-defined test data; 3: Create real processing tasks.
DstResources.N No Array of DataTransformResouceInfo Destination topic_id and alias of processing task. This parameter is required when FuncType=1, and not required when FuncType=2.
EnableFlag No Integer Task status. Valid values: 1 (enabled) and 2 (disabled).
PreviewLogStatistics.N No Array of PreviewLogStatistic Test data used for previewing the processing result

3. Output Parameters

Parameter Name Type Description
TaskId String Task ID
RequestId String The unique request ID, generated by the server, will be returned for every request (if the request fails to reach the server for other reasons, the request will not obtain a RequestId). RequestId is required for locating a problem.

4. Example

Example1 Creating a Data Processing Task

This example shows you how to create a data processing task. Similar to the open source component Logstash, data processing provides capabilities such as filtering, cleaning, anonymization, enrichment, and distribution of log data. Domain Specific Language (DSL) functions are implemented based on Flink for data processing, allowing you to easily process log stream data.

Input Example

POST / HTTP/1.1
Host: cls.tencentcloudapi.com
Content-Type: application/json
X-TC-Action: CreateDataTransform
<Common request parameters>

{
    "EnableFlag": 1,
    "Name": "Data processing task",
    "FuncType": 0,
    "PreviewLogStatistics": [
        {
            "LineNum": 0,
            "FailReason": "errorcode",
            "DstTopicId": "81XXXXe5-e39e-4a1e-b2d4-a778df97d825",
            "LogContent": "XXXX",
            "Time": "2017-08-08 12:12:12"
        }
    ],
    "SrcTopicId": "xxxx-xx-xx-xx-xxxxxxxx",
    "TaskType": 0,
    "DstResources": [
        {
            "TopicId": "3d9bXXXX-05a4-4435-ac65-f43e684329b3",
            "Alias": "Alias"
        }
    ],
    "EtlContent": "xx-xx"
}

Output Example

{
    "Response": {
        "TaskId": "xxxx-xx-xx-xx-xxxxxxxx",
        "RequestId": "6ef60bec-0242-43af-bb20-270359fb54a7"
    }
}

5. Developer Resources

SDK

TencentCloud API 3.0 integrates SDKs that support various programming languages to make it easier for you to call APIs.

Command Line Interface

6. Error Code

The following only lists the error codes related to the API business logic. For other error codes, see Common Error Codes.

Error Code Description
FailedOperation Operation failed.
InternalError Internal error.
InvalidParameter Incorrect parameter.
InvalidParameter.DataFromTaskConflict There is a data processing task conflict.
InvalidParameter.InvalidEtlContent The data processing statement is invalid.
MissingParameter Missing parameter.
OperationDenied Operation denied.
OperationDenied.ACLFailed ACL verification failed.
OperationDenied.AccountDestroy The account has been terminated.
OperationDenied.AccountIsolate The account has overdue payments.
OperationDenied.AccountNotExists The account does not exist.
ResourceNotFound The resource does not exist.
ResourceNotFound.TopicNotExist The log topic does not exist.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon