tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Tencent Cloud WeData
    Documentation
    Download PDF

    Conversion Node Configuration

    Last updated: 2024-11-01 17:05:06
    Download PDF

      Nodes related to Field Conversion

      String Replacement

      The string replacement node mainly uses Java regular expressions to match and replace the content of string fields. This node will perform the replacement operation when the string is partially or completely matched in the field content. The configuration parameters for conversion rules are as follows:
      Parameters
      Description
      Field
      Select the field to match the string. The fields processed by this node must be of string type.
      Note:
      In this conversion node, multiple conversion rules can be configured for the same field. Multiple rules will be serialized according to the configuration order.
      Regular Expression
      Java regular expression used to match the original field content.
      New string
      Replace the matched field content with the specified new string.
      Replacement method
      Supports replacing only the first match or all matches.
      Delete
      Delete this row rule.

      Field Segmentation

      The field splitting node is primarily used when the content of the original field is divided using a fixed symbol. The divided content is then written into a new field, while the original content remains unchanged. This node is often used in scenarios where field content undergoes one-to-many mapping, such as dividing the content in field A "Tencent-cloud" using "-" and writing it into two new fields B and C. The conversion rule parameters are as follows:
      Parameters
      Description
      Field
      Select the field that needs content segmentation.
      Note:
      In this conversion node, multiple conversion rules can be configured for the same field.
      Delimiter
      Fill in the segmentation identifier.
      Note:
      The delimiter can be chosen from the dropdown menu or manually entered. You can select built-in delimiters from the dropdown menu or fill them in manually.
      Result Field Name
      Input the field name that will receive the segmented results. Use a comma to separate multiple field names. The segmented results will be filled into the target fields in sequence by default. If the number of target field names exceeds the segmented results, the extra fields will have a default empty value. Conversely, any excess segmented content will be discarded.
      Note:
      If field A (containing "Tencent-cloud") is divided using "-" and written into fields B, C, and D, then fields B and C will be filled with Tencent and cloud, respectively, while the content of field D will be the default empty value.
      Delete
      Delete this row rule.
      Note:
      Currently, this node only supports real-time synchronization tasks.

      Nodes related to Data Cleaning

      Data Filtering

      Filter and match the contents of each row in the table according to the filtering rules. For matched data rows, you can either retain or remove them.\\nConversion rules and parameter descriptions are as follows:
      Parameters
      Description
      Filtering Action
      Retain: Write entries that match the filtering rules into the target table.
      Remove: Do not write data that hits the rule to the target table.
      Field
      Select the field that needs content segmentation.
      Note:
      In this conversion node, multiple conversion rules can be configured for the same field.
      Logical Operator
      Support AND, OR.
      Operator
      Currently supports >, <, <=, >=, =, !=, is null, and is not null.
      Type
      Value type used for comparing with field content:
      Field: Use the specified field's content to filter field comparison, typically varying with different data rows.
      Custom value: Use a fixed constant to compare with the filter field content.
      Comparison value
      Choose a field or enter a custom value.
      Delete
      Delete this row rule.
      Note:
      Currently, this node only supports real-time synchronization tasks.

      Deduplication

      Deduplicate data within the window based on real-time data processing time (process_time). The transformation rule parameters are described as follows:
      Parameters
      Description
      Deduplication action
      Retain the first item: When duplicate data exists in the time window, retain the first item in the time sequence.
      Retain the last item: When duplicate data exists in the time window, retain the last item in the time sequence.
      Deduplication field
      Specify the field used to determine whether the field content is duplicated. If unspecified, all fields are used by default.
      Delete
      Delete this row rule.
      Note:
      Currently, this node only supports real-time synchronization tasks.

      Data connection(join)

      Real-time data stream join, currently supporting regular join. Regular join is suitable for bounded input streams, and defaults to saving all state. Join nodes by default require configuration of two input nodes. The conversion rules parameters are explained as follows:
      Parameters
      Description
      Left/Right Table
      Select the name of the node in the upstream join input node to serve as the left table object.
      Left/Right Table Associated Key
      The field name used for join association in the left and right tables.
      Connection Method
      Supports Left Join, Right Join, and Full Join.
      
      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy