Tencent Cloud

Recent Pages

JSONPath Description

Last updated: 2024-11-07 11:39:19

Overview
JSON is currently one of the most commonly used format protocols for Internet information transmission. Data processing mainly focus on parsing and handling JSON data.
JSONPath is a message query syntax specification designed for the JSON format. For data processing, simple JSONPath syntax can be used to quickly retrieve the value of a member in a complex nested JSON struct, and extension functions in the JayWay library can also be used to aggregate or operate on a certain type of member fields.
Basic Features
Basic Syntax
$ is a root node operator and represents the root node of the current JSON struct.
.<childName> is a dot operator, and ['<childName>'] is a bracket operator. They represent the selected child member named childName of the current object.
.. is a recursive operator and indicates recursively obtaining all child members of the current object.
[<index>] is a selection operator and indicates obtaining the No.index child member of the current iterable object.
Obtaining Specific Member Variable of Nested JSON Struct
The following figure shows the structure of container standard output logs collected by TKE:
{
  "@timestamp": 1648803500.63659,
  "@filepath": "/var/log/tke-log-agent/test7/xxxxxxxx-adfe-4617-8cf3-9997aea90ded/c_tke-es-xxxxxxxx57-n29jr_default_nginx-xxxxxxxx49626ef42d5615a636aae74d6380996043cf6f6560d8131f21a4d8ba/jgw_INFO_2022-02-10_15_4.log",
  "log": "15:00:00.000[4349811564226374227] [http-nio-8081-exec-64] INFO  com.qcloud.jgw.gateway.server.topic.TopicService",
  "kubernetes": {
    "pod_name": "tke-es-xxxxxxxxxx-n29jr",
    "namespace_name": "default",
    "pod_id": "xxxxxxxx-adfe-4617-8cf3-9997aea90ded",
    "labels": {
      "k8s-app": "tke-es",
      "pod-template-hash": "xxxx95d557",
      "qcloud-app": "tke-es"
    },
    "annotations": {
      "qcloud-redeploy-timestamp": "1648016531476",
      "tke.cloud.tencent.com/networks-status": "[{\\n    \\"name\\": \\"tke-bridge\\",\\n    \\"interface\\": \\"eth0\\",\\n    \\"ips\\": [\\n        \\"172.16.x.xx\\"\\n    ],\\n    \\"mac\\": \\"xx:xx:xx:4a:c2:ba\\",\\n    \\"default\\": true,\\n    \\"dns\\": {}\\n}]"
    },
    "host": "10.0.xx.xx",
    "container_name": "nginx",
    "docker_id": "xxxxxxxx49626ef42d5615a636aae74d6380996043cf6f6560d8131f21a4d8ba",
    "container_hash": "nginx@sha256:xxxxxxxx7b29b585ed1aee166a17fad63d344bc973bc63849d74c6452d549b3e",
    "container_image": "nginx"
  }
}
When users need to obtain the current Pod name, that is, the qcloud-app member field, they can use the $.kubernetes.labels.qcloud-app or $.['kubernetes'].['labels'].['qcloud-app'] JSONPath syntax.
The running result is as follows. The figure shows that the corresponding logs have been successfully read using JSONPath:
﻿
﻿
Note
When JSONPath is used to handle parameters, only the square bracket operator can be used if a JSON variable name contains special characters such as ..
For example, for a JSON struct like {"key1.key2":"value1"}, $.['key1.key2'] must be used to obtain the corresponding member fields.
Advanced Features
Advanced Syntax
* is a wildcard operator and indicates obtaining all child objects of the current object.
*~ is a built-in function and indicates obtaining the names of all child objects of the current iterable object.
min() is a built-in function and indicates obtaining the minimum value of child objects of the current iterable object.
max() is a built-in function and indicates obtaining the maximum value of child objects of the current tterable object.
sum() is a built-in function and indicates obtaining the sum of child objects of the current iterable object.
concat() is a built-in function that concatenates multiple objects into a string.
Aggregating Data of Specific Fields
When there is an object list in a JSON struct, the list length is usually variable. Take the request response log in the figure below as an example:
{
  "data": {
    "Response": {
      "Result": {
        "Routers": [
          {
            "AccessType": 0,
            "RouteId": 81111,
            "VpcId": "vpc-xxxxxxxx",
            "VipType": 3,
            "VipList": [
              {
                "Vip": "10.0.0.189",
                "Vport": "9xxx"
              }
            ]
          },
          {
            "AccessType": 0,
            "RouteId": 81112,
            "VpcId": "vpc-r5sbavzp",
            "VipType": 3,
            "VipList": [
              {
                "Vip": "10.0.0.248",
                "Vport": "9xxx"
              }
            ]
          },
          {
            "AccessType": 0,
            "RouteId": 81113,
            "VpcId": "vpc-xxxxxxxx",
            "VipType": 3,
            "VipList": [
              {
                "Vip": "10.0.0.210",
                "Vport": "9xxx"
              }
            ]
          }
        ]
      },
      "RequestId": "20e74750-ca40-403d-9ea9-d3f63b5415d2"
    }
  },
  "code": 0
}
When you need to aggregate the member attributes of a variable-length list, you cannot use the processing chain for aggregation. Instead, you can use the JSONPath syntax * to match all elements in the list.
For example, if you want to obtain all the Vip elements in VipList, you can use the JSONPath syntax $.data.Response.Result.Routers[*].VipList[0].Vip.
The running result is shown below. It shows that all Vip elements in the struct are successfully obtained.
﻿
﻿
Concatenating and Modify Struct Members
In some scenes, multiple objects in a JSON struct need to be concatenated during data processing to facilitate transfer to the downstream for further operations. The following syntax can be used:
{
  "data": {
    "Response": {
      "SubnetSet": [
        {
          "VpcId": "vpc-xxxxxxxx",
          "SubnetId": "subnet-xxxxxxxx",
          "SubnetName": "ckafka_cloud_subnet-1",
          "CidrBlock": "10.0.0.0/19",
          "Ipv6CidrBlock": "",
          "IsDefault": false,
          "IsRemoteVpcSnat": false,
          "EnableBroadcast": false,
          "Zone": "ap-changsha-ec-1",
          "RouteTableId": "rtb-xxxxxxxx",
          "NetworkAclId": "",
          "TotalIpAddressCount": 8189,
          "AvailableIpAddressCount": 8033,
          "CreatedTime": "2021-01-25 17:31:00",
          "TagSet": [],
          "CdcId": "",
          "IsCdcSubnet": 0,
          "LocalZone": false,
          "IsShare": false
        }
      ],
      "TotalCount": 1,
      "RequestId": "705c4955-0cd9-48b2-9132-79eadae2e3e6"
    }
  },
  "code": 0
}
If the downstream has no computing capability, and the VPC and subnet attributes need to be concatenated during data processing, the JSONPath function concat() can be used to concatenate multiple fields and modify the output string.
For example, the syntax $.concat($.data.Response.SubnetSet[0].VpcId,"#",$.data.Response.SubnetSet[0].SubnetId,"#",$.data.Response.SubnetSet[0].CidrBlock)) can be used to concatenate the VPC and subnet attributes, which can be separated with the character #.
The running result is shown below. It shows that the information on concatenated VPC resources are successfully obtained:
﻿
﻿
﻿

Contact Us

Contact our sales team or business advisors to help your business.

Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

7x24 Phone Support

tencent cloud

Recent Pages

JSONPath Description

Overview

Basic Features

Basic Syntax

Obtaining Specific Member Variable of Nested JSON Struct

Advanced Features

Advanced Syntax

Aggregating Data of Specific Fields

Concatenating and Modify Struct Members

Was this page helpful?

Was this page helpful?

tencent cloud

Sign Up

Log in

Recent Pages

JSONPath Description

Overview

Basic Features

Basic Syntax

Obtaining Specific Member Variable of Nested JSON Struct

Advanced Features

Advanced Syntax

Aggregating Data of Specific Fields

Concatenating and Modify Struct Members

Was this page helpful?

Was this page helpful?