Kubernetes can efficiently improve business orchestration capabilities and resource utilization. With no additional capabilities for support, this enhancement remains substantially limited. The average resource utilization of a TKE node is merely about 14% according to the previous statistics by the TKE team.
The main reason for the poor resource utilization rate of a Kubernetes cluster is adherence to Kubernetes' resource scheduling logic. When creating Kubernetes workloads, it's typical to configure suitable resource Requests and Limits for the workload, indicating resource concession and restriction. Among these, the Requests have the most significant impact on the utilization rate. To prevent the resources employed by their workload from being occupied by others, or to cater to the resource demands during peak traffic, users tend to set larger values for Request. The disparity between the Requests and the actual utilized resources cannot be employed by other workloads, resulting in wastage. The unreasonable setting of Request values leads to a low resource utilization rate in the Kubernetes cluster.
Tencent Kubernetes Engine (TKE) supports the installation of Request Recommendation component in the cluster. Request Recommendation allows for the suggestion of Request/Limit values for container-level resources in Kubernetes workloads, reducing resource wastage.
Resource objects deployed in a cluster
By enabling Request Recommendation in a cluster, it will deploy the following Kubernetes objects within a cluster:
Kubernetes Object Name
Type
Default Resource Occupation
Associated Namespace
analytics.analysis.crane.io
CustomResourceDefinition
-
-
recommendations.analysis.crane.io
CustomResourceDefinition
-
-
crane-system
Namespace
-
-
housekeeper-default
Analytics
-
crane-system
recommendation-config
ConfigMap
-
crane-system
craned
ClusterRole
-
-
craned
ClusterRoleBinding
-
-
craned
Service
-
crane-system
craned
ServiceAccount
-
crane-system
craned
Deployment
-
crane-system
Feature Overview
It supports recommending suitable Request/Limit values of resources for each container in Deployment, StatefulSet, and DaemonSet.
It supports one-click update of the resource values for containers in the initial workload with recommended values.
It supports maintaining the Request/Limit ratio. The recommended Request/Limit will preserve the proportion between the Reqeust/Limit in the initial Workload Container Settings. If the Limit is not set upon Workload creation, a Limit recommendation won't be provided.
The console's one-click update capability for Request recommendations will add a nodeSelector attribute to the workload by default. During workload updates, Pods can only be scheduled on native nodes. If native node resources are insufficient, it will lead to a pending of the Pod.
Principles of Request Recommendation
The component creates an Analytics CR object under the crane-system Namespace, covering all native Kubernetes workloads (Deployment, DaemonSet, StatefulSet) in all clusters. It analyzes workload data for up to 14 days, updating recommended values every 12 hours.
It then produces a Recommendation CR object for each workload within the cluster based on Analytics, purposed for data storage of recommendations.
If recommendation CR generates recommendation data, it will inscribe this information into the corresponding workload's Annotation.
Notes
Environment Requirements
Kubernetes version: 1.10+
Node Requirements
The One-Click Update Workload Request feature in the Tencent Kubernetes Engine Console will migrate the workload to the native node. If your cluster's native node lacks resources, it could result in a pending of the Pod.
Requirements on the Controlled Resources
It supports Deployment, StatefulSet, and DaemonSet.
It does not support Job and CronJob, as well as the Pods that are not managed by a workload.
Recommended Threshold
Suggested minimum values: The recommended minimum value for CPU per container is 0.125 core, i.e. 125 m; the minimum memory is 125 Mi.
You can also undertake the installation in TKE Insight > Workload Map.
3. On the Node Map page, hover your mouse over a Node at the bottom of the page, and click Details.
4. In the top right corner of the Node details page, enable the Request Recommendation switch to configure the scheduler's parameters.
Note:
This feature comprises a global switch at the cluster level. After the feature is enabled, it will automatically analyze the historical monitoring data of workloads to recommend appropriate Request values.
This feature does not take effect immediately after enabling. The system will analyze the resource usage history to provide accurate recommended values.
The period for calculation may vary for different workloads. One workload within a cluster may potentially impact another.
After this feature is enabled, values will be recommended for the workloads that run at least for one day.
For workloads created after this feature is enabled, it usually takes one day to recommend values.
It is recommended to update the Workload with the recommended values after the workload has been running stably for a while.
Workload Map mainly displays various states and metrics of workloads through a visual interface, assisting users in comprehending the current configuration volume of the workload and its actual usage, thereby helping in analyzing potential issues within the workload. For more information, see the Workload Map documentation.
3. On the Workload Map page, hover your mouse over a workload at the bottom of the page, and click Recommended.
4. In the pop-up window, click Confirm to use the suggested Request value for updating the original value in the Workload.
Note:
The One-Click Update Workload Request feature in the Tencent Kubernetes Engine Console will migrate the workload to the native nodes. If your native nodes in your cluster lack resources, it will result in a pending of the Pod.
Accessing Recommended Values in the Background
The Request Recommendation engine stores the recommended values in the YAML file of each workload. You can use the standard Kubernetes API to access these recommended values for each workload and then integrate them into your business's deployment system. The following demonstrates how to peek into the recommended Request amount for each container under a workload:
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
analysis.crane.io/resource-recommendation:|
containers:
# If a Pod contains multiple containers, each container has recommended values for CPU and Memory Request
- containerName: nginx
target:
cpu: 125m
memory: 125Mi #If unit is missing herein, a character string "58243235" will be displayed, with byte as the omitted unit
Note:
The component itself does not recommend a Limit. When updating the Workload using the Request recommendation value in the console, it will maintain the ratio of the Workload’s Request and Limit to ensure the Quality of Service (QoS) remains constant. If you access the recommended value of the Request in the background, you can consider it as a reference to update the resource configuration of the original Workload.
Component Permission Description
Permission Description
The permission of this component is the minimal dependency required for the current feature to operate.
Permission Scenarios
Feature
Involved Object
Involved Operation Permission
Recording the oom record of the pod
pod
get/list/watch
Searching and recommending idle nodes based on the node
node
get/list/watch
It is required to record the exception information in the form of events.
event
create/patch/update
Monitoring changes in related recommendation resources, and making resource recommendation
Yes
No
Was this page helpful?