Operation Scenarios
This document introduces how to configure and optimize Nginx Ingress for high concurrency scenarios.
Operation Guide
Increasing CLB Specifications and Bandwidth
High concurrency scenarios require high traffic throughput and forwarding performance of CLB. You can manually create a CLB in the CLB Console, select LCU-supported instance specifications, choose the model as needed, and increase the bandwidth limit (ensure the VPC is consistent with that of the TKE cluster). After the CLB is created, configure Nginx Ingress to reuse this CLB as the traffic entry. For details, refer to Custom Definition CLB. Tuning Kernel Parameters and Nginx Configuration
Optimize kernel parameters and Nginx configuration for high concurrency scenarios. values.yaml
configuration method:
controller:
extraInitContainers:
- name: sysctl
image: busybox
imagePullPolicy: IfNotPresent
securityContext:
privileged: true
command:
- sh
- -c
- |
sysctl -w net.core.somaxconn=65535 # Increase connection queue to prevent queue overflow.
sysctl -w net.ipv4.ip_local_port_range="1024 65535" # Expand the source port range to prevent port exhaustion.
sysctl -w net.ipv4.tcp_tw_reuse=1 # Enable TIME_WAIT reuse to allow new connections after port exhaustion.
sysctl -w fs.file-max=1048576 # Increase the file handle count to prevent connection overflow from exhausting file handles.
config:
keep-alive-requests: "1000"
upstream-keepalive-connections: "2000"
max-worker-connections: "65536"
Log Rotation
Nginx Ingress will print logs to the container's standard output by default, which will be managed automatically by the container during running. In high-concurrency scenarios, this may lead to high CPU occupancy.
The solution is to output Nginx Ingress logs to log files and use a sidecar to automatically rotate the log files, preventing the disk space from being filled up with logs.
values.yaml
configuration method:
controller:
config:
access-log-path: /var/log/nginx/nginx_access.log
error-log-path: /var/log/nginx/nginx_error.log
extraVolumes:
- name: log
emptyDir: {}
extraVolumeMounts:
- name: log
mountPath: /var/log/nginx
extraContainers:
- name: logrotate
image: imroc/logrotate:latest
imagePullPolicy: IfNotPresent
env:
- name: LOGROTATE_FILE_PATTERN
value: "/var/log/nginx/nginx_*.log"
- name: LOGROTATE_FILESIZE
value: "100M"
- name: LOGROTATE_FILENUM
value: "3"
- name: CRON_EXPR
value: "*/1 * * * *"
- name: CROND_LOGLEVEL
value: "8"
volumeMounts:
- name: log
mountPath: /var/log/nginx
Was this page helpful?