tencent cloud

Why is the cluster in an exceptional status?

In the following conditions, the cluster will be in the red or yellow status:

• If the cluster has any unassigned primary index shard, the cluster status will become red, which affects index reads/writes and thus requires special attention.
• If all primary index shards in the cluster have been assigned, but there are still unassigned replica index shards, the cluster status will become yellow, which does not affect index reads/writes and generally can be automatically recovered.

Viewing Cluster Status

You can use Kibana Dev Tools to view the cluster status:

GET /_cluster/health

Here, you can see that the current cluster status is red, and there are nine unassigned shards.

Official descriptions of the Elasticsearch health API responses:

Troubleshooting

If a cluster is exceptional, you need to pay attention to the shards that are not assigned properly in unassigned_shards. The following is an example:

Finding exceptional index

View the index status and find the exceptional index based on the response.

GET /_cat/indices

Viewing exception details

GET /_cluster/allocation/explain

Through the exception information, you can see that:

1. The primary shard is currently in the unassigned status (current_state). This problem occurs because the node to which the shard was assigned left the cluster (unassigned_info.reason).
2. After the above problem occurs, the shard cannot be automatically assigned because there are no available replicas of the shard in the cluster (can_allocate).
3. In addition, more detailed information is provided (allocate_explanation).

This problem occurs because the cluster has an offline node, so the primary shard has no available shard data. Currently, the only thing you can do is to wait for the node to recover and join the cluster again.

Note：

In some extreme cases (for example, a shard in a single-replica cluster is corrupted, or the file system fails, causing the node to be removed permanently), you can only accept the fact of data loss and use the reroute command to assign an empty primary shard again. To avoid such cases as much as possible, we recommend you appropriately design index shards and refrain from setting a single replica for the index (single replica is also called zero replica, which means that an index has a primary shard but no replica shards). With the appropriate design of index shards, you can control the total number of shards in the cluster at a healthy scale, make better use of the distributed cluster characteristics while ensuring high availability, and improve the overall cluster performance.

All possible reasons of unassigned shards (unassigned_info.reason)

You can use the following analysis methods to preliminarily figure out the reason why there is an unassigned shard in the cluster. Generally, you can find the reason through the allocation explain API.

Note：

If the cluster status hasn't automatically recovered after a long period of time, or you cannot fix the problem, please submit a ticket for assistance.