tencent cloud

Creating rollup task

In scenarios with massive amounts of data, a business system can generate petabytes of data per day or even per hour. Time series data is generally massive in amount, time-sensitive, and trending. Therefore, systems using such data (e.g., monitoring or data analysis systems) often only need high-precision data in the most recent time period while downsampling historical data for storage. You can configure a rollup task to periodically aggregate historical data and save it into a new metric. A rollup task not only saves downsampled historical data, but also improves query performance and reduces storage costs. Note that a rollup task automatically creates child metrics inheriting all configurations of the parent metric according to base_metric, and the parent metric configurations will be overwritten if options is specified.

1. Request address

The address is the instance IP and port, such as 10.13.20.15:9200, which can be obtained in the console.

2. Request path and method

Path: /_rollup/${rollup_task_name}, where ${rollup_task_name} is the rollup task name.
Method: PUT

3. Request parameters

None

4. Request content

5. Response content

You need to judge whether a request is successful based on the error field. If the response content contains the error field, the request failed. For the error details, see the error field description.

6. Sample code for curl

Request:

curl -u root:le201909 -H 'Content-Type:application/json' -X POST 172.xx.xx.4:9201/_rollup/ctsdb_rollup_task_test -d'
{
 "base_metric": %{base_metric_name},      
 "rollup_metric": %{rollup_metric_name},    
 "base_rollup": %{base_rollup_name},        
 "query" : "name:host AND type:max",   
 "group_by": ["host"],   
 "function": {
     "cost_total":  {
         "sum": { 
             "field": "cost"
         }
     },
     "cpu_usage_avg": {
         "avg": {
             "field": "cpu_usage"
         }
     },
     "value": {   
       "percentiles": {      
         "field": "value",
         "percents": [
           95
         ]
       }
     },
     "metricName": {
       "set": {              
         "value": "cpu_usage"
       }
     },
     "appid": {
       "any": {             
         "field": "appid"
       }
     },
      "first_value": { 
       "first": {           
         "field": "value"
       }
     },
      "last_value": {
       "last": {             
         "field": "value"
       }
     }
 },
 "interval": "1m",           
 "frequency": "5m",          
 "delay": "1m",             
 "start_time": "1502892000",
 "end_time": "2147483647",
 "options": {
     "expire_day": 365
 }
}
'

Response:

{
 "acknowledged": true,
 "message": "create rollup success"
}

Getting names of all rollup tasks

1. Request address

The address is the instance IP and port, such as 10.13.20.15:9200, which can be obtained in the console.

2. Request path and method

Path: /_rollups
Method: GET

3. Request parameters

None

4. Request content

None

5. Response content

You need to judge whether a request is successful based on the error field. If the response content contains the error field, the request failed. For the error details, see the error field description.

6. Sample code for curl

Request:
curl -u root:le201909 -H 'Content-Type:application/json' -X GET 172.xx.xx.4:9201/_rollups

Response:

{
 "result": 
{
  "rollups": 
  [
      "rollup_jgq_6",
      "rollup_jgq_60"
  ]
 },
 "status": 200
}

Getting rollup task details

1. Request address

The address is the instance IP and port, such as 10.13.20.15:9200.

2. Request path and method

Path: /_rollup/${rollup_task_name}, where ${rollup_task_name} is the rollup task name.
Method: GET

3. Request parameters

You can specify the v parameter to view the specific rollup progress. @last_end_time in the response structure is the latest rollup progress.

4. Request content

None

5. Response content

You need to judge whether a request is successful based on the error field. If the response content contains the error field, the request failed. For the error details, see the error field description.

6. Sample code for curl

Request:
curl -u root:le201909 -H 'Content-Type:application/json' -X GET 172.xx.xx.4:9201/_rollup/rollup_jgq_6?v

Response:

{
"result": {
 "rollup_jgq_6": {
 "base_metric": "cvm_device-300",
    "rollup_metric": "cvm_device-86400",
    "query": "metricName:cpu_usage AND statType:max",
    "group_by": [
      "vm_uuid"
    ],
    "function": {
      "value": {
        "percentiles": {
          "field": "value",
          "percents": [
            95
          ]
        }
      },
      "metricName": {
        "set": {
          "value": "cpu_usage"
        }
      },
      "appid": {
        "any": {
          "field": "appid"
        }
      }
    },
    "interval": "1d",
    "delay": "5m",
    "options": {
      "expire_day": 186
    },
    "frequency": "1d",
    "start_time": 1534003200,
    "end_time": 2147483647,
    "@state": "running",             // Running status
    "@timestamp": 1550766085000,     // Rollup task information update time point
    "@last_end_time": 1550764800     // Rollup task end time point for proper execution
  }
},
"status": 200
}

Deleting rollup task

1. Request address

The address is the instance IP and port, such as 10.13.20.15:9200.

2. Request path and method

Path: /_rollup/${rollup_task_name}, where ${rollup_task_name} is the rollup task name.
Method: DELETE

3. Request parameters

None

4. Request content

None

5. Response content

You need to judge whether a request is successful based on the error field. If the response content contains the error field, the request failed. For the error details, see the error field description.

6. Sample code for curl

Request:
curl -u root:le201909 -H 'Content-Type:application/json' -X DELETE 172.xx.xx.4:9201/_rollup/ctsdb_rollup_task_test

Response:

{
 "acknowledged": true,
 "message": "delete rollup success"
}

Updating rollup task

1. Request address

The address is the instance IP and port, such as 10.13.20.15:9200.

2. Request path and method

Path: /_rollup/${rollup_task_name}/update, where ${rollup_task_name} is the rollup task name.
Method: POST

3. Request parameters

None

4. Request content

5. Response content

You need to judge whether a request is successful based on the error field. If the response content contains the error field, the request failed. For the error details, see the error field description.

6. Sample code for curl

Request:

curl -u root:le201909 -H 'Content-Type:application/json' -X POST 172.xx.xx.4:9201/_rollup/ctsdb_rollup_task_test/update -d'
{
 "state":"running",
 "start_time": "1511918989",
 "end_time": "1512019765",
 "options": 
{
     "expire_day": 365
 }
}'

Response:

{
 "acknowledged": true,
 "message": "update rollup success"
}