By default, a Service configures the NodePort of a cluster node as the CLB backend, as shown in the TKE Access Layer Components section below. This solution is highly fault-tolerant, where traffic from a CLB instance to any NodePort will be forwarded to a random Pod. This is also the most basic network access layer solution proposed by Kubernetes, as shown below:
TKE Service Controller
does not use the following nodes as the CLB backend by default:
NotReady
status (unhealthy nodes).Note:
TKE Service Controller
can bind nodes in theUnschedulable
status. They can be used as the traffic ingress, as they will forward the received traffic in the container network and will not discard it, as shown above.
For some very large clusters, a Service-managed CLB instance will mount the NodePort of almost all cluster nodes as the backend, which may cause the following problems:
Such problems can be solved in the following ways:
For some large clusters, you can specify some nodes to be bound by using the service.kubernetes.io/qcloud-loadbalancer-backends-label
annotation, which contains a label selector that allows you to bind matching nodes after they are labeled. This process is synced, which means when a node changes so that it is selected or no longer selected, TKE Service Controller
will add or remove the corresponding backend on the CLB instance. For more information, see Labels and Selectors.
service.kubernetes.io/qcloud-loadbalancer-backends-label
does not select any node, the Service backend will be emptied, interrupting the service. This feature requires cluster node label management.Deploy a test application containing only one or two Pods under a large cluster. When the service is exposed through Service, the CLB instance will perform a health check on all the backend NodePorts, and the number of such requests has a huge impact on the test application. In this case, you can specify a small number of nodes in the cluster as the backends by using labels to relieve the pressure brought by health checks. For more information, see High Health Check Frequency.
apiVersion: v1
kind: Service
metadata:
annotations:
service.kubernetes.io/qcloud-loadbalancer-backends-label: "group=access-layer"
name: nginx-service
spec:
ports:
- name: 80-80-no
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
This sample contains the following configuration:
service.kubernetes.io/qcloud-loadbalancer-backends-label
annotation declares the backend selector, and only cluster nodes labeled group=access-layer
will be used as the CLB backends.Kubernetes provides the ExternalTrafficPolicy
Service feature. When it is set to Local
, traffic will not be forwarded between nodes through NAT, reducing NAT operations and retaining the source IP. NodePort will only forward traffic to the Pod of the current node. The Local
mode has the following characteristics:
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
externalTrafficPolicy: Local
ports:
- name: 80-80-no
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
By default, when the Local mode is enabled for a Service, the NodePorts of almost all nodes will be mounted as the backends. The CLB instance will not forward traffic to backend nodes without workloads based on the health check result. To prevent backends without workloads from being bound, you can specify nodes with workloads as the backends in Local mode by using the service.kubernetes.io/local-svc-only-bind-node-with-pod: "true"
annotation. For more information, see Using Source IP.
apiVersion: v1
kind: Service
metadata:
annotations:
service.kubernetes.io/local-svc-only-bind-node-with-pod: "true"
name: nginx-service
spec:
externalTrafficPolicy: Local
ports:
- name: 80-80-no
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
In Local mode, the request traffic to a node is not forwarded among nodes. Therefore, when nodes have different numbers of workloads, using the same backend weight may make the load on each node uneven. In this case, you can perform weighted balancing by using the service.cloud.tencent.com/local-svc-weighted-balance: "true"
annotation, where the weight of the NodePort backend will be determined by the number of workloads on the node, thus avoiding load unevenness caused by the different numbers of workloads on different nodes. Here, Local weighted balancing must be used in conjunction with Local binding as shown below:
apiVersion: v1
kind: Service
metadata:
annotations:
service.kubernetes.io/local-svc-only-bind-node-with-pod: "true"
service.cloud.tencent.com/local-svc-weighted-balance: "true"
name: nginx-service
spec:
externalTrafficPolicy: Local
ports:
- name: 80-80-no
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
Was this page helpful?