tencent cloud

Background

On a single ES node, the FST structure in the inverted index resides permanently in the heap memory, which uses a relatively high proportion of the memory (up to over 50% especially on cold nodes with large disks). This restricts the capability of a single node to manage disks, limits the heap memory, and thus affects the node availability. As there are very few query requests on cold nodes, storing FST in the memory is of little significance. Therefore, we need to move this part of data structure out of the heap for management, so that it is not loaded by default but loaded from the disk off heap for direct use when needed, reducing the heap memory usage and improving the single-node disk management capabilities.

Optimized Scheme

ES implements a precisely controlled off-heap cache based on the WLFU elimination strategy and implements a second-level in-heap cache based on zero copy and weak reference. In this way, the performance is comparable to that of in-heap access.

Directions

Enabling/Disabling off-heap feature (disabled by default)

curl -H "Content-Type:application/json" -XPUT http://localhost:9200/_cluster/settings -d '{
  "persistent" : {
    "indices.segment_memory.off_heap.enable" : true
   }
}'

Adjusting the size of off-heap cache (500 MB by default)

curl -H "Content-Type:application/json" -XPUT http://localhost:9210/_cluster/settings -d '{
 "persistent" : {
    "indices.segment_memory.off_heap.size" : "5gb"
  }
}'

It can be set to 1/3 of the off-heap memory of a single node and should not exceed 32 GB. Below are specific examples:

• If the total memory of s single node (including JVM and off-heap memory) is 64 GB, it can be set to (64 – 32)/3. You can set it to 10 GB.
• If the total memory of s single node (including JVM and off-heap memory) is 96 GB, it can be set to (96 – 32)/3. You can set it to 20 GB.

Optimization Effect

The optimized scheme works much better in terms of memory overheads, data management, and GC and has a slightly better performance.

Supported Versions

6.8.2, 7.5.1, and 7.10.1