tencent cloud

Feedback

Implementing Horizontal Scaling based on CLB monitoring metrics using KEDA in TKE

Last updated: 2024-05-31 09:43:18

    Business scenario

    Business traffic on TKE usually enters through CLB (Tencent Cloud Load Balancer). Sometimes, you may want the workload to scale based on CLB's monitoring metrics, for example:
    1. Long connection scenarios (such as game rooms and online meetings): Each user corresponds to one connection, and the maximum number of connections handled by each pod in the workload is relatively fixed. In this case, scaling can be performed based on the CLB connection count metric.
    2. Online services using the HTTP protocol: The metric Queries Per Second (QPS) that a single pod in the workload can support is relatively fixed. In this case, scaling can be performed based on the CLB QPS metric.

    Introduction to keda-tencentcloud-clb-scaler

    KEDA has many built-in triggers, but not for Tencent Cloud CLB. However, KEDA supports external triggers for expansion. The keda-tencentcloud-clb-scaler is a KEDA External Scaler based on Tencent Cloud CLB monitoring metrics, capable of auto scaling based on metrics such as CLB connection count, QPS, and bandwidth.

    Directions

    Installing keda-tencentcloud-clb-scaler

    helm repo add clb-scaler https://imroc.github.io/keda-tencentcloud-clb-scaler
    helm upgrade --install clb-scaler clb-scaler/clb-scaler -n keda \\
    --set region="ap-chengdu" \\
    --set credentials.secretId="xxx" \\
    --set credentials.secretKey="xxx"
    Replace the region with the one where your CLB instance is located (usually the same as the cluster region). For details on the region list, refer to Regions and availability zones.
    The credentials.secretId and credentials.secretKey are the key pair of your Tencent Cloud account, used to access and retrieve CLB monitoring data. Replace them with your own key pair.

    Deploying workload

    You can use the following workload YAML sample for testing:
    apiVersion: v1 kind: Service metadata: labels: app: httpbin name: httpbin spec: ports: - port: 8080 protocol: TCP targetPort: 80 selector: app: httpbin type: LoadBalancer --- apiVersion: apps/v1 kind: Deployment metadata: name: httpbin spec: replicas: 1 selector: matchLabels: app: httpbin template: metadata: labels: app: httpbin spec: containers: - image: kennethreitz/httpbin:latest name: httpbin
    After the workload is deployed, a corresponding public network CLB instance will be automatically created to manage traffic. You can run the following command to obtain the CLB ID:
    $ kubectl svc httpbin -o jsonpath='{.metadata.annotations.service\\.kubernetes\\.io/loadbalance-id}' lb-********
    Record the obtained CLB ID, which will be used in subsequent KEDA configurations.

    Using ScaledObject to configure auto scaling based on CLB monitoring metrics

    Configuration method

    CLB-based monitoring metrics are commonly used for online services. Configure auto scaling using KEDA's ScaledObject, set the trigger type to external, and input the required metadata, mainly including the following fields:
    The scalerAddress is the address used by keda-operator to call keda-tencentcloud-clb-scaler.
    The loadBalancerId is the ID of the CLB instance.
    The metricName is the name of the CLB monitoring metric. Most metrics are the same for public and private networks.
    The threshold is the metric threshold for scaling up or down, meaning that scaling-up or scaling-down is performed by comparing the value of metricValue/number of pods to the threshold.
    The listener is the unique optional configuration, specifying the CLB listener for monitoring metrics, in the format of protocol/port.

    Configuration example 1: Auto scaling based on the metric CLB connection count

    apiVersion: keda.sh/v1alpha1 kind: ScaledObject metadata: name: httpbin spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: httpbin pollingInterval: 15 minReplicaCount: 1 maxReplicaCount: 100 triggers: - type: external metadata: # highlight-start scalerAddress: clb-scaler.keda.svc.cluster.local:9000 loadBalancerId: lb-xxxxxxxx metricName: ClientConnnum # Connection count threshold: "100" # 100 connections per pod listener: "TCP/8080" # Optional. Specifies the listener in the format of protocol/port. # highlight-end

    Configuration example 2: Auto scaling based on the metric CLB QPS

    apiVersion: keda.sh/v1alpha1 kind: ScaledObject metadata: name: httpbin spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: httpbin pollingInterval: 15 minReplicaCount: 1 maxReplicaCount: 100 triggers: - type: external metadata: # highlight-start scalerAddress: clb-scaler.keda.svc.cluster.local:9000 loadBalancerId: lb-xxxxxxxx metricName: TotalReq # Requests per second threshold: "500" # Average 500 QPS per pod listener: "TCP/8080" # Optional. Specifies the listener in the format of protocol/port. # highlight-end
    
    
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support