Function | Description | Syntax Description | Return Value Type |
ext_sep | Extracts field value content based on a separator. | ext_sep("Source field name", "Target field 1,Target field 2,Target field...", sep="Separator", quote="Non-segmentation part"", restrict=False, mode="overwrite") | Log after extraction (LOG) |
ext_sepstr | Extracts field value content based on specified characters (string). | ext_sepstr("Source field name","Target field 1,Target field 2,Target field...", sep="abc", restrict=False, mode="overwrite") | Log after extraction (LOG) |
ext_json | Extracts field values from JSON data. | Log after extraction (LOG) ext_json("Source field name",prefix="",suffix="",format="full",exclude_node="JSON nodes not to expand") | Log after extraction (LOG) |
ext_json_jmes | Extracts a field value based on a JMES expression. | ext_json_jmes("Source field name", jmes= "JSON extraction expression", output="Target field", ignore_null=True, mode="overwrite") | Log after extraction (LOG) |
ext_kv | Extracts field values by using two levels of separators. | ext_kv("Source field name", pair_sep=r"\\s", kv_sep="=", prefix="", suffix="", mode="fill-auto") | Log after extraction (LOG) |
ext_regex | Extracts field values by using a regular expression. | ext_regex("Source field name", regex="Regular expression", output="Target field 1,Target field 2,Target field...", mode="overwrite") | Log after extraction (LOG) |
ext_first_notnull | Returns the first non-null and non-empty result value. | ext_first_notnull(Value 1,Value 2,...) | The first non-null result value |
Function | Description | Syntax Description | Return Value Type |
enrich_table | Uses CSV structure data to match fields in logs and, when matched fields are found, the function adds other fields and values in the CSV data to the source logs. | enrich_table("CSV source data", "CSV enrichment field", output="Target field 1,Target field 2,Target field....", mode="overwrite") | Mapped log (LOG) |
enrich_dict | Uses dict structure data to match a field value in a log. If the specified field and value match a key in the dict structure data, the function assigns the value of the key to another field in the log. | enrich_dict("JSON dictionary", "Source field name", output=Target field name, mode="overwrite") | Mapped log (LOG) |
Function | Description | Syntax Description | Return Value Type |
compose | Combines multiple operation functions. Providing combination capabilities similar to those of branch code blocks, this function can combine multiple operation functions and execute them in sequence. It can be used in combination with branches and output functions. | compose("Function 1","Function 2", ...) | Log (LOG) |
t_if | Executes a corresponding function if a condition is met and does not perform any processing if the condition is not met. | t_if("Condition", Function) | Log (LOG) |
t_if_not | Executes a corresponding function if a condition is not met and does not perform any processing if the condition is met. | t_if_not("Condition",Function) | Log (LOG) |
t_if_else | Executes a function based on the evaluation result of a condition. | t_if_else("Condition", Function 1, Function 2) | Log (LOG) |
t_switch | Executes different functions depending on whether branch conditions are met. If all conditions are not met, the data is deleted. | t_switch("Condition 1", Function 1, "Condition 2", Function 2, ...) | Log (LOG) |
Function | Description | Syntax Description | Return Value Type |
log_output | Outputs a row of log to a specified log topic. This function can be used independently or together with branch conditions. | log_output(Log topic alias) (The alias here is the target log topic alias specified when the data processing task is created.) | No return value |
log_split | Splits a row of log into multiple rows of logs based on the value of a specified field by using a separator and JMES expression. | log_split(Field name, sep=",", quote="\\"", jmes="", output="") | Log (LOG) |
log_drop | Deletes logs that meet a specified condition. | log_drop(Condition 1) | Log (LOG) |
log_keep | Retains logs that meet a specified condition. | log_keep(Condition 1) | Log (LOG) |
log_split_jsonarray_jmes | Splits and expands the JSON array in the log according to JMES syntax. | log_split_jsonarray_jmes("field", jmes="items", prefix="") | Log (LOG) |
Function | Description | Syntax Description | Return Value Type |
fields_drop | Deletes the fields that meet a specified condition. | fields_drop(Field name 1, Field name 2, ..., regex=False,nest=False) | Log (LOG) |
fields_keep | Retains the fields that meet a specified condition. | fields_keep(Field name 1, Field name 2, ..., regex=False) | Log (LOG) |
fields_pack | Matches field names based on a regular expression and encapsulates the matched fields into a new field whose value is in JSON format. | fields_pack(Target field name, include=".*", exclude="", drop_packed=False) | Log (LOG) |
fields_set | Sets field values or adds fields. | fields_set(Field name 1, Field value 1, Field name 2, Field value 2, mode="overwrite") | Log (LOG) |
fields_rename | Renames fields. | fields_rename(Field name 1, New field name 1, Field name 2, New field name 2, regex=False) | Log (LOG) |
has_field | If the specified field exists, returns `True`. Otherwise, returns `False`. | has_field(Field name) | Condition value (BOOL) |
not_has_field | If the specified field does not exist, returns `True`. Otherwise, returns `False`. | not_has_field(Field name) | Condition value (BOOL) |
v | Gets the value of a specified field and returns the corresponding string. | v(Field name) | Value string type (STRING) |
Function | Description | Syntax Description | Return Value Type |
json_select | Extracts a JSON field value with a JMES expression and returns the JSON string of the extraction result. | json_select(v(Field name), jmes="") | Value string type (STRING) |
xml_to_json | Parses and converts an XML-formatted value to a JSON string. The input value must be an XML string. Otherwise, a conversion exception will occur. | xml_to_json(Field value) | Value string type (STRING) |
json_to_xml | Parses and converts a JSON string value to an XML string. | json_to_xml(Field value) | Value string type (STRING) |
if_json | Checks whether a value is a JSON string. | if_json(Field value) | Condition value (BOOL) |
Function | Description | Syntax Description | Return Value Type |
regex_match | Matches data in full or partial match mode based on a regular expression and returns whether the match is successful. | regex_match(Field value, regex="", full=True) | Condition value (BOOL) |
regex_select | Matches data based on a regular expression and returns the corresponding partial match result. You can specify the sequence number of the matched expression and the sequence number of the group to return (partial match + sequence number of the specified matched group). If no data is matched, an empty string is returned. | regex_select(Field value, regex="", index=1, group=1) | Value string type (STRING) |
regex_split | Splits a string and returns a JSON array of the split strings (partial match). | regex_split(Field value, regex=\\"\\", limit=100) | Value string type (STRING) |
regex_replace | Matches data based on a regular expression and replaces the matched data (partial match). | regex_replace(Field value, regex="", replace="", count=0) | Value string type (STRING) |
regex_findall | Matches data based on a regular expression and returns a JSON array of the matched data (partial match). | regex_findall(Field value, regex="") | Value string type (STRING) |
Function | Description | Syntax Description | Return Value Type |
dt_str | Converts a time field value (a date string in a specific format or timestamp) to a target date string of a specified time zone and format. | dt_str(Value, format="Formatted string", zone="") | STRING |
dt_to_timestamp | Converts a time field value (a date string in a specified format; time zone specified) to a UTC timestamp. | dt_to_timestamp(Value, zone="") | STRING |
dt_from_timestamp | Converts a timestamp field value to a time string in the specified time zone. | dt_from_timestamp(Value, zone="") | STRING |
dt_now | Obtains the current datetime of the processing calculation. | dt_now(format="Formatted string", zone="") | STRING |
Function | Description | Syntax Description | Return Value Type |
str_count | Searches for a substring in a specified range of a value and returns the number of occurrences of the substring. | str_count(Value, sub="", start=0, end=-1) | INT |
str_len | Returns the length of a string. | str_len(Value) | INT |
str_uppercase | Converts a string to uppercase. | str_uppercase(Value) | STRING |
str_lowercase | Converts a string to lowercase. | str_lowercase(Value) | STRING |
str_join | Concatenates input values by using a concatenation string. | str_join(Concatenation string 1, Value 1, Value 2, ...) | STRING |
str_replace | Replaces an old string with a new string. | str_replace(Value, old="", new="", count=0) | STRING |
str_format | Formats strings. | str_format(Formatting string, Value 1, Value 2, ...) | STRING |
str_strip | Deletes specified characters from a string concurrently from the start and end of the string, and returns the remaining part. | str_strip(Value, chars="\\t\\r\\n") | STRING |
str_lstrip | Deletes specified characters from a string from the start of the string, and returns the remaining part. | str_strip(Value, chars="\\t\\r\\n") | STRING |
str_rstrip | Deletes specified characters from a string from the end of the string, and returns the remaining part. | str_strip(Value, chars="\\t\\r\\n") | STRING |
str_find | Checks whether a string contains a specified substring and returns the position of the first occurrence of the substring in the string. | str_find(Value, sub="", start=0, end=-1) | INT |
str_start_with | Checks whether a string starts with a specified string. | str_start_with(Value, sub="", start=0, end=-1) | BOOL |
str_end_with | Checks whether a string ends with a specified string. | str_end_with(Value, sub="", start=0, end=-1) | BOOL |
Function | Description | Syntax Description | Return Value Type |
op_if | Returns a value based on a specified condition. | op_if(Condition 1, Value 1, Value 2) | If the condition is `true`, `Value 1` is returned; otherwise, `Value 2` is returned. |
op_and | Performs the AND operation on values. If all the specified parameter values are evaluated to true, `True` is returned; otherwise, `False` is returned. | op_and(Value 1, Value 2, ...) | BOOL |
op_or | Performs the OR operation on values. If one or more of the specified parameter values are evaluated to false, `False` is returned; otherwise, `True` is returned. | op_or(Value 1, Value 2, ...) | BOOL |
op_not | Performs the NOT operation on values. | op_not(Value) | BOOL |
op_eq | Compares two values. If the values are equal, `True` is returned. | op_eq(Value 1, Value 2) | BOOL |
op_ge | Compares two values. If `Value 1` is greater than or equal to `Value 2`, `True` is returned. | op_ge(Value 1, Value 2) | BOOL |
op_gt | Compares two values. If `Value 1` is greater than `Value 2`, `True` is returned. | op_gt(Value 1, Value 2) | BOOL |
op_le | Compares two values. If `Value 1` is less than or equal to `Value 2`, `True` is returned. | op_le(Value 1, Value 2) | BOOL |
op_lt | Compares two values. If `Value 1` is less than `Value 2`, `True` is returned. | op_lt(Value 1, Value 2) | BOOL |
op_add | Returns the sum of two specified values. | op_add(Value 1, Value 2) | Calculation result |
op_sub | Returns the difference between two specified values. | op_sub(Value 1, Value 2) | Calculation result |
op_mul | Returns the product of two specified values. | op_mul(Value 1, Value 2) | Calculation result |
op_div | Returns the quotient of two specified values. | op_div(Value 1, Value 2) | Calculation result |
op_sum | Returns the sum of multiple specified values. | op_sum(Value 1, Value 2, ...) | Calculation result |
op_mod | Returns the remainder of a specified value divided by the other specified value. | op_mod(Value 1, Value 2) | Calculation result |
op_null | Checks whether a value is `null`. If so, `true` is returned; otherwise, `false` is returned. | op_null(Value) | BOOL |
op_notnull | Checks whether a value is not `null`. If so, `true` is returned; otherwise, `false` is returned. | op_notnull(Value) | BOOL |
op_str_eq | Compares string values. If they are equal to each other, `true` is returned. | op_str_eq(Value 1, Value 2, ignore_upper=False) | BOOL |
Function | Description | Syntax Description | Return Value Type |
ct_int | Converts a value (whose base can be specified) to a decimal integer. | ct_int(Value 1, base=10) | Calculation result |
ct_float | Converts a value to a floating-point number. | ct_float(Value) | Calculation result |
ct_str | Converts a value to a string. | ct_str(Value) | Calculation result |
ct_bool | Converts a value to a Boolean value. | ct_bool(Value) | Calculation result |
Function | Description | Syntax Description | Return Value Type |
decode_url | Decodes an encoded URL. | decode_url(Value) | STRING |
Function | Description | Syntax Description | Return Value Type |
geo_parse | Parses the geographical location. | geo_parse(Field value, keep=("country","province","city"), ip_sep=",") | JSON string |
Was this page helpful?