tencent cloud

All product documents
Cloud Log Service
DocumentationCloud Log ServiceOperation GuideData Processing documentsData ProcessingData Processing FunctionsRow Processing Functions
Row Processing Functions
Download PDF
Last updated: 2024-01-20 17:44:35
Row Processing Functions
Last updated: 2024-01-20 17:44:35
Download PDF

Overview

Row processing functions process log rows, such as filtering, distributing, and splitting log rows.



Function log_output

Function definition

This function is used to output a row of log to a specified log topic. It can be used independently or together with branch conditions.

Syntax description

log_output(Alias). The alias is defined when the processing task is configured.

Field description

Parameter
Description
Type
Required
Default Value
Value Range
alias
Log topic alias
string
Yes
-
-

Sample

Distribute the log to 3 different log topics according to the values (waring, info, and error) of the loglevel field.
Raw log:
[
  {
    "loglevel": "warning"
  },
  {
    "loglevel": "info"
  },
  {
    "loglevel": "error"
  }
]
Processing rule:
// The `loglevel` field has 3 values (`waring`, `info`, and `error`) and therefore the log is distributed to 3 different log topics accordingly.
t_switch(regex_match(v("loglevel"),regex="info"),log_output("info_log"),regex_match(v("loglevel"),regex="warning"),log_output("warning_log"),regex_match(v("loglevel"),regex="error"),log_output("error_log"))

Function log_split

Function definition

This function is used to split a row of log into multiple rows of logs based on the value of a specified field by using a separator and JMES expression.

Syntax description

log_split(Field name, sep=",", quote="\"", jmes="", output="")

Field description

Parameter
Description
Type
Required
Default Value
Value Range
field
Field to extract
string
Yes
-
-
sep
Separator
string
No
,
Any single character
quote
Characters that enclose the value
string
No
-
-
jmes
JMES expression. For more information, see JMESPath.
string
No
-
-
output
Name of a single field
string
Yes
-
-

Sample

Example 1. Split a log whose field has multiple values
{"field": "hello Go,hello Java,hello python","status":"500"}
Processing rule:
// Use the separator "," to split the log into 3 logs.
log_split("field", sep=",",  output="new_field")
Processing result:
{"new_field":"hello Go","status":"500"}
{"new_field":"hello Java","status":"500"}
{"new_field":"hello python","status":"500"}
Example 2. Use a JMES expression to split a log
{"field": "{\"a\":{\"b\":{\"c\":{\"d\":\"a,b,c\"}}}}", "status": "500"}
Processing rule:
// The value of `a.b.c.d` is `a,b,c`.
log_split("field", jmes="a.b.c.d",  output="new_field")
Processing result:
{"new_field":"a","status":"500"}
{"new_field":"b","status":"500"}
{"new_field":"c","status":"500"}
Example 3. Split a log that contains a JSON array
{"field": "{\"a\":{\"b\":{\"c\":{\"d\":[\"a\",\"b\",\"c\"]}}}}", "status": "500"}
Processing rule:
log_split("field", jmes="a.b.c.d",  output="new_field")
Processing result:
{"new_field":"a","status":"500"}
{"new_field":"b","status":"500"}
{"new_field":"c","status":"500"}

Function log_drop

Function definition

This function is used to delete logs that meet a specified condition.

Syntax description

log_drop(Condition 1)

Field description

Parameter
Description
Type
Required
Default Value
Value Range
condition
Function expression whose return value is of bool type
bool
Yes
-
-

Sample

Delete logs where status is 200 and retain other logs.
Raw log:
{"field": "a,b,c", "status": "500"}
{"field": "a,b,c", "status": "200"}
Processing rule:
log_drop(op_eq(v("status"), 200))
Processing result:
{"field":"a,b,c","status":"500"}

Function log_keep

Function definition

This function is used to retain logs that meet a specified condition.

Syntax description

log_keep(Condition 1)

Field description

Parameter
Description
Type
Required
Default Value
Value Range
condition
Function expression whose return value is of bool type
bool
Yes
-
-

Sample

Retain logs where status is 500 and delete other logs.
Raw log:
{"field": "a,b,c", "status": "500"}
{"field": "a,b,c", "status": "200"}
Processing rule:
log_keep(op_eq(v("status"), 500))
Processing result:
{"field":"a,b,c","status":"500"}

Function log_split_jsonarray_jmes

Function definition

This function is used to split and expand the JSON array in the log according to JMES syntax.

Syntax description

log_split_jsonarray_jmes("field", jmes="items", prefix="")

Field description

Parameter
Description
Type
Required
Default Value
Value Range
field
Field to extract
string
Yes
-
-

Sample

Example 1 Raw log:
{"common":"common","result":"{\"target\":[{\"a\":\"a\"},{\"b\":\"b\"}]}"}
Processing rule:
log_split_jsonarray_jmes("result",jmes="target")
fields_drop("result")
Processing result:
{"common":"common", "a":"a"}
{"common":"common", "b":"b"}
Example 2 Raw log:
{"common":"common","target":"[{\"a\":\"a\"},{\"b\":\"b\"}]"}
Processing rule:
log_split_jsonarray_jmes("target",prefix="prefix_")
fields_drop("target")
Processing result:
{"prefix_a":"a", "common":"common"}
{"prefix_b":"b", "common":"common"}

Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 available.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon