This API can be used to write data records into a single metric either in bulk or not. To improve the write efficiency, we recommend you write data in bulk.
The address is the instance IP and port, such as 10.13.20.15:9200
, which can be obtained in the console.
Request path: /${metric_name}/doc/_bulk
, where ${metric_name}
is the metric name.
Method: POST
Note:The
doc
keyword is the_type
of the written data and must be added to facilitate subsequent system parsing and upgrade.
You can set the filter_path
parameter to filter and simplify the returned results. For more information, please see Batch Querying Data.
You need to bulk write data as structured data in NDJSON format into a metric, which is similar to the following:
Metadata\n
Data to be written\n
....
Metadata\n
Data to be written\n
The format of the metadata is as shown below:
{
"index" :
{
"_id" : "1", # Document ID (optional)
"_routing": "sh" # Routing value (optional)
}
}
The format of the written data is as shown below:
{
"field1" : "value1",
"field2" : "value2"
}
Note:When writing data, you can set the
_routing
parameter to specify the shards where data to be written. This parameter is optional and can be specified to any value. You can specify the same_routing
value for different data records to route them into the same shard. In addition, if you specify an already set_routing
parameter value during a query, the system will query data in the specified shard, which can greatly accelerate the query. You need to add a line break at the end of the request body.
You should note that the returned result of the bulk data write API is different from those of other APIs. Please pay attention to the errors
(not error
) field in the JSON result first. If it is false
, all data records were successfully written; if it is true
, some data records failed to be written, and you can get the failure details from the items
field.
The items
field is an array, where each element corresponds to a write request. You can check whether each element has the error
field to judge whether the corresponding request is successful. If the error
field exists, the request failed. The specific error information is in the error
field. If the error
field doesn't exist, the request succeeded.
Sample response upon success:
Request:
curl -u root:le201909 -H 'Content-Type:application/json' -X POST 172.16.345.14:9201/ctsdb_test/doc/_bulk -d'
{"index":{"_routing": "sh" }}
{"region":"sh","cpuUsage":2.5,"timestamp":1505294654}
{"index":{"_routing": "sh" }}
{"region":"sh","cpuUsage":2.0,"timestamp":1505294654}
'
Response:
{
"took": 134,
"errors": false,
"items":
[
{
"index":
{
"_index": "ctsdb_test@1505232000000_1",
"_type": "doc",
"_id": "AV_8eeo_UAkC9PF9L-2q",
"_version": 1,
"result": "created",
"_shards":
{
"total": 2,
"successful": 2,
"failed": 0
},
"created": true,
"status": 201
}
},
{
"index":
{
"_index": "ctsdb_test2@1505232000000_1",
"_type": "doc",
"_id": "AV_8eeo_UAkC9PF9L-2r",
"_version": 1,
"result": "created",
"_shards":
{
"total": 2,
"successful": 2,
"failed": 0
},
"created": true,
"status": 201
}
}
]
}
Note:The
errors
value returned above isfalse
, indicating that all data records were successfully written. Theitems
array indicates the write result of each record in the same order as in thebulk
request. For single records initems
, thestatus
of2XX
indicates that the record was successfully written._index
indicates the child metric where the data was written, and_shards
indicates the replica write status. In the above sample,total
indicates two replicas, andsuccessful
indicates that data was successfully written into both replicas.
Sample response upon failure:
Request:
curl -u root:le201909 -H 'Content-Type:application/json' -X POST 172.16.345.14:9201/hcbs_client_trace/doc/_bulk -d'
{"index":{"_type":"type"}}
{"vol_id":"c57e008c-0ae0-41cd-8da8-6989d0522fc6","io_type":2,"data_len":4096,"latency":3,"try_times":1,"errcode":0,"start_time":1503404266}
{"index":{"_type":"type"}}
{"vol_id":"c57e008c-0ae0-41cd-8da8-6989d0522fc6","io_type":"abc","data_len":4096,"latency":1,"try_times":1,"errcode":0,"start_time":1503404266}
'
Response:
{
"took": 7,
"errors": true,
"items": [
{
"index": {
"_index": "hcbs_client_trace",
"_type": "type",
"_id": "AWMe9r9lNifptzIWMVPT",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"created": true,
"status": 201
}
},
{
"index": {
"_index": "hcbs_client_trace",
"_type": "type",
"_id": "AWMe9r9lNifptzIWMVPU",
"status": 400,
"error": {
"type": "mapper_parsing_exception",
"reason": "failed to parse [io_type]",
"caused_by": {
"type": "number_format_exception",
"reason": "For input string: \"abc\""
}
}
}
}
]
}
Note:The
errors
value returned above istrue
, indicating that some data records failed to be written. Theitems
array indicates the write result of each record in the same order as in thebulk
request. For single records initems
, thestatus
of2XX
indicates that the record was successfully written. Theerror
field indicates the detailed error information._index
indicates the child metric where the data was written, and_shards
indicates the replica write status. In the above sample,total
indicates two replicas, andsuccessful
indicates that data was successfully written into both replicas.
This API can be used to write data records into multiple metrics either in bulk or not. To improve the write efficiency, we recommend you write data in bulk.
The address is the instance IP and port, such as 10.13.20.15:9200
, which can be obtained in the console.
Request path: _bulk
Method: PUT
Note:The
doc
keyword is the_type
of the written data and must be added to facilitate subsequent system parsing and upgrade.
You can set the filter_path
parameter to filter and simplify the returned results. For more information, please see Batch Querying Data.
You need to bulk write data as structured data in NDJSON format into a metric, which is similar to the following:
Metadata\n
Data to be written\n
....
Metadata\n
Data to be written\n
The format of the metadata is as shown below:
{
"index" :
{
"_index" :"metric_name", # Metric where data is to be written
"_type" : "doc", # Document type of the data to be written
"_id" : "1", # Document ID (optional)
"_routing": "sh" # Routing value (optional)
}
}
The format of the written data is as shown below:
{
"field1" : "value1",
"field2" : "value2"
}
Note:When writing data, you can set the
_routing
parameter to specify the shards where data to be written. This parameter is optional and can be specified to any value. You can specify the same_routing
value for different data records to route them into the same shard. In addition, if you specify an already set_routing
parameter value during a query, the system will query data in the specified shard, which can greatly accelerate the query. Note that you need to add a line break at the end of the request body.
You need to judge whether a request is successful based on the error
field. If the response content contains the error
field, the request failed. The specific error information is in the error
field. Note: if the request succeeded but the errors
(not error
) field is not false
, the specific data that failed to be written is indicated in the errors
field.
Request:
curl -u root:le201909 -H 'Content-Type:application/json' -X PUT 172.16.345.14:9201/_bulk -d'
{"index":{"_index" : "ctsdb_test", "_type" : "doc", "_routing": "sh" }}
{"region":"sh","cpuUsage":2.5,"timestamp":1505294654}
{"index":{"_index" : "ctsdb_test2", "_type" : "doc", "_routing": "sh" }}
{"region":"sh","cpuUsage":2.0,"timestamp":1505294654}
'
Response:
{
"took": 134,
"errors": false,
"items":
[
{
"index":
{
"_index": "ctsdb_test@1505232000000_1",
"_type": "doc",
"_id": "AV_8eeo_UAkC9PF9L-2q",
"_version": 1,
"result": "created",
"_shards":
{
"total": 2,
"successful": 2,
"failed": 0
},
"created": true,
"status": 201
}
},
{
"index":
{
"_index": "ctsdb_test2@1505232000000_1",
"_type": "doc",
"_id": "AV_8eeo_UAkC9PF9L-2r",
"_version": 1,
"result": "created",
"_shards":
{
"total": 2,
"successful": 2,
"failed": 0
},
"created": true,
"status": 201
}
}
]
}
Note:The
errors
value returned above isfalse
, indicating that all data records were successfully written. Theitems
array indicates the write result of each record in the same order as in thebulk
request. For single records initems
, thestatus
of2XX
indicates that the record was successfully written._index
indicates the child metric where the data was written, and_shards
indicates the replica write status. In the above sample,total
indicates two replicas, andsuccessful
indicates that data was successfully written into both replicas.
Was this page helpful?