tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Tencent Kubernetes Engine
    Documentation
    Download PDF

    Update Notes of TKE Kubernetes Major Versions

    Last updated: 2024-05-28 10:25:38
    Download PDF

      1.26 changes since 1.24

      Major Updates

      PodSecurityPolicy has been removed, and Pod Security Admission becomes stable: PodSecurityPolicy was deprecated in version 1.21 and completely removed in version 1.25. The necessary updates to improve its usability would result in breaking changes, thus it was necessary to remove it and replace it with the more user-friendly Pod Security Admission. If you are currently using PodSecurityPolicy, please see Migrating from PodSecurityPolicy to the built-in Pod Security admission.
      Ephemeral Containers are now stable: Ephemeral containers were in beta in version 1.23 and have become stable in version 1.25. They can be used in a pod for inspection and troubleshooting when a container cannot be effectively kubectl exec'd due to a crash or the lack of debugging tools in the image.
      Cgroups v2 has become stable in version 1.25: Compared with cgroups v1, cgroups v2 offers many improvements. While cgroups v1 will continue to be supported, this cgroups v2 enhancement readies Kubernetes for eventually deprecating and switching to v2.
      Windows continues to be optimized in version 1.25, such as support for unit tests, consistency tests, and creating a new repository for Windows Operational Readiness.
      The image repository has been migrated from k8s.gcr.io to registry.k8s.io.
      SeccompDefault enters the beta phase in version 1.25.
      NetWorkPolicy's endPort becomes stable in version 1.25. Previously, each NetworkPolicy only supported a single port, while the endPort field allows for specifying a range of ports.
      Local Ephemeral Storage Capacity Isolation becomes stable in version 1.25, providing capacity isolation for local ephemeral container storage between pods, such as EmptyDir. If a pod's consumption of local ephemeral capacity storage exceeds the limit, eviction can be used to forcefully limit its consumption of shared resources.
      CSI Migration becomes stable in version 1.25. The CSI migration is a continuous effort made by SIG Storage over multiple versions before, with the purpose of migrating in-tree volume plugins to out-of-tree CSI drivers.
      CSI Ephemeral Volume becomes stable in version 1.25. The CSI ephemeral volume feature allows for directly specifying CSI volumes in the Pod specification for ephemeral use scenarios. They can be used to inject any state (such as configuration, secrets, identities, variables, or similar information) directly into a Pod through mounted volumes. This was initially introduced as an alpha feature in version 1.15 and has been upgraded to the official version. Some CSI drivers (such as the secret-store CSI driver) have used this feature.
      CRD Validation Expression Language enters the beta phase in version 1.25.
      Server Side Unknown Field Validation enters the beta phase in version 1.25, enabled by default. The apiserver supports validation for unknown fields, allowing for the planned, sequential removal of client-side validation features in kubectl.
      KMS v2 alpha1 API is introduced in version 1.25 to enhance performance, implement rotation, and realize observable improvements. Use AES-GCM instead of AES-CBC, and implement static data encryption through DEK without requiring additional actions from users, while continuing to allow reading using both AES-GCM and AES-CBC.
      CRI v1alpha2 API is removed in version 1.26, and CRI v1 version is recommended. Version 1.26 will not support containerd 1.5 and earlier versions. If you are using containerd, it is required to upgrade to containerd v1.6.0 or higher before upgrading the node to Kubernetes v1.26.
      In the v1beta1 Traffic Control API Group in version 1.26: FlowSchema and PriorityLevelConfiguration's flowcontrol.apiserver.k8s.io/v1beta1 API version will no longer be available in v1.26. The flowcontrol.apiserver.k8s.io/v1beta2 API version is available since v1.23.
      Starting with 1.25, the HorizontalPodAutoscaler of the autoscaling/v2beta1 API version will no longer be provided, and the HorizontalPodAutoscaler of the autoscaling/v2beta2 API version will no longer be available in version 1.26. It is recommended to use the autoscaling/v2 API version, available since v1.23.
      The in-tree credential management code, originally as a part of Kubernetes and authentication code specific to vendors Azure and Google Cloud, will be removed from client-go and kubectl. As an alternative, Kubernetes has provided a vendor-neutral authentication plugin mechanism before the release of v1.26.

      Deprecation and Removal

      Parameter

      kubectl Command-line Parameter: As part of the implementation of the Inclusive Naming Initiative, in version 1.26, the --prune-whitelist flag will be deprecated and replaced with --prune-allowlist.
      kube-apiserver Command-line Parameter: The --master-service-namespace command-line parameter has no effect on kube-apiserver and has been informally deprecated. This command-line parameter will be officially marked as deprecated in v1.26, with plans for future removal. The Kubernetes project is not expected to be affected by this deprecation and removal.
      kubectl run Command-line Parameters: In 1.26, several unused option parameters for the kubectl run sub-command will be marked as deprecated, including: cascade, filename, force, grace-period, kustomize, recursive, timeout, and wait.
      The kube-proxy's userspace mode is removed in 1.26. Linux users shall use iptables or ipvs, while Windows users shall use . Now using --mode userspace will fail.

      API Version

      1.25
      batch/v1beta1 has been removed, and batch/v1 is used instead.
      discovery.k8s.io/v1beta1 has been removed, and discovery.k8s.io/v1 is used instead.
      events.k8s.io/v1beta1 has been removed, and events.k8s.io/v1 is used instead.
      policy/v1beta1 has been removed, and policy/v1 is used instead.
      node.k8s.io/v1beta1 has been removed, and node.k8s.io/v1 is used instead.
      autoscaling/v2beta1 has been removed, and autoscaling/v2 is used instead.
      1.26
      autoscaling/v2beta2 has been removed, and autoscaling/v2 is used instead.
      flowcontrol.apiserver.k8s.io/v1beta1 has been removed, and flowcontrol.apiserver.k8s.io/v1beta2 and flowcontrol.apiserver.k8s.io/v1beta3 can be used instead.

      Others

      1.25 deprecates the GlusterFS and Portworx in-tree volume plugins. Although a CSI driver was built for GlusterFS, it has not been maintained. The community discussed the possibility of migrating to a compatible CSI driver but ultimately decided to start deprecating the GlusterFS plugin from the in-tree drivers.
      1.25 removes PodSecurityPolicy, Flocker, Quobyte, StorageOS.
      1.25 change in vSphere version support: The in-tree vSphere volume driver will no longer support any vSphere versions earlier than 7.0u2.
      1.25 cleaning ownership of IPTables chain: Kubelet will gradually migrate to not creating the following iptables chains in the nat table: KUBE-MARK-DROP, KUBE-MARK-MASQ and KUBE-POSTROUTING. This change will be implemented in phases through the IPTablesCleanup feature gate.
      1.26 removes the GlusterFS in-tree driver, removal of the deprecated OpenStack in-tree storage integration (Cinder volume type).

      Changelogs

      

      1.24 changes since 1.22

      Note:
      1. Kubernetes 1.24 has removed the support for Docker through Dockershim. For new nodes in TKE, only Containerd 1.6.9 is supported upon the container runtime .
      2. For clusters upgrading from 1.22 to 1.24, nodes whose runtime versions are docker or containerd 1.4 or later (excluding 1.4) can only be upgraded through reinstalling the rolling upgrade mode.

      Major Update

      Remove Dockershim from kubelet

      The built-in dockershim of kubelet was deprecated in v1.20 and removed starting from v1.24. From v1.24 onwards, nodes using the Docker runtime need to migrate to its runtime, such as containerd or CRI-O. For more information, refer to Is Your Cluster Ready for v1.24.

      Upgrade EphemeralContainers to Beta

      The ephemeral container feature entered Beta in 1.23. Ephemeral containers exist in a Pod for a limited time. They are particularly useful for troubleshooting when you need to inspect another container that has either crashed or cannot use kubectl exec because its image lacks the necessary debugging tools.

      API with Beta disabled by default

      New beta version APIs, starting from 1.24, are disabled by default. Existing beta version and upgraded Beta version APIs are not affected.

      Storage capacity and volume expansion are upgraded to GA

      This feature is GA in 1.24.Storage capacity tracking is supported through CSIStorageCapacity Objects to obtain the current available storage capacity, enhancing the scheduling of Pods with delayed binding CSI volumes.
      Volume Expansion increased support for resizing existing persistent volumes.

      NonPreemptingPriority Feature GA

      NonPreemptingPriority feature is GA in 1.24, enabled by default and cannot be disabled.

      GRPC-based detection upgraded to Beta

      GRPCContainerProbe feature is upgraded to beta in 1.24 and enabled by default.

      IdentifyPodOS feature upgraded to Beta

      The IdentifyPodOS feature is introduced in 1.23 and upgraded to Beta in 1.24. Once enabled, the podSpec can specify an os field, indicating the desired OS type (linux/windows) for the Pod to run on. When creating a Pod, this field will be used to validate other Pod settings, such as the SecurityContext field. Note, it is still necessary to select nodes of the expected OS type for the Pod using the nodeSelector kubernetes.io/os. If the OS of the scheduled node does not match the OS specified in this field, the kubelet will refuse to run the Pod.

      JobReadyPods feature enters Beta

      The JobReadyPods feature introduced in 1.23 is upgraded to Beta in 1.24. Once enabled, the number of Ready Pods will be displayed in the Job status.

      KubeletCredentialProviders feature upgraded to Beta

      This feature is in 1.24 Beta and enabled by default, allowing kubelet to dynamically obtain credentials for container image repositories using the exec plugin, rather than storing credentials on the node's file system.

      ServiceIPStaticSubrange alpha Feature

      The ServiceIPStaticSubrange feature is introduced in 1.24. Once enabled, it can effectively prevent conflicts with the ClusterIPs allocated to services.

      Dynamic Kubelet configuration has been removed

      After being deprecated in 1.22, the Dynamic Kubelet configuration was removed in 1.24.

      Incompatible changes related to CNI versions

      The following runtime versions have been verified to be compatible with 1.24:
      containerd v1.6.4+,v1.5.11+
      CRI-O 1.24+
      For containerd versions v1.6.0 to v1.6.3, it's necessary to upgrade the CNI plugin or declare the CNI configuration version; otherwise, errors such as "Incompatible CNI versions" or "Failed to destroy network for sandbox" may occur.

      CSI snapshot v1beta1 was removed

      VolumeSnapshot v1beta1 was deprecated in v1.20 and removed in 1.24.

      Deprecate FlexVolume

      Starting from 1.23, FlexVolume is deprecated, and the use of CSI for developing volume drivers is recommended.

      Deprecate parameters related to klog

      <0>Starting from 1.23, all klog parameters are deprecated except for <1>--v</1><2> and </2><3>--vmodule</3><4>: --log-dir, --log-file, --log-flush-frequency, --logtostderr, --alsologtostderr, --one-output, --stderrthreshold, --log-file-max-size, --skip-log-headers, --add-dir-header, --skip-headers, --log-backtrace-at</4>

      Upgrade of IPv4/IPv6 Dual Stack Support to Stable

      Starting from k8s 1.21, dual stack support is enabled by default, entering GA in 1.23, and the IPv6DualStack feature has been removed. Although the cluster supports dual stack networks by default, pods and services are still single-stack by default. To use dual stack, nodes must have routable IPv4/IPv6 network interfaces and use a dual stack-supported CNI network plugin. Also, the Service must be configured to use dual stack: The Service's .spec.ipFamilyPolicy field should be set to PreferDualStack or RequireDualStack . For more information, refer to IPv4/IPv6 dual-stack.

      HorizontalPodAutoscaler v2 enters GA

      autoscaling/v2 API enters GA in 1.23, deprecating autoscaling/v2beta2 API.

      Generic Ephemeral Volume enters GA

      The Ephemeral Volume feature enters GA in 1.23, allowing the use of all storage plugins that support dynamic creation to provide ephemeral volumes for Pods.

      Ignore volume ownership change to enter GA

      This feature enters GA in 1.23, allowing for setting the .securityContext.fsGroupChangePolicy field to OnRootMismatch . Owner and access permissions of the content inside are changed only if the root directory's Owner and access permissions do not match the expected permissions of the volume. Otherwise, it allows skipping the recursive setting of file permissions when mounting a volume, and speeding up Pod startup.

      PodSecurity upgraded to Beta

      PodSecurity, intended to replace the deprecated PodSecurityPolicy Admission Controller, enters the Beta phase in 1.23.

      Default to using CRI v1

      Starting from 1.23, kubelet supports and defaults to using the CRI v1 API. If v1 is not supported at runtime, it will fall back to v1alpha2.

      Structured logging upgraded to Beta

      Structured logging is upgraded to Beta in 1.23, with most logs from kubelet and kube-scheduler recorded in structured JSON format.

      Simplified Scheduler Plugin Configuration

      Starting from 1.23, the MultiPoint option is introduced in the Scheduler Plugins configuration. Plugins configured here are automatically registered to all supported extension point plugin collections, simplifying plugin configuration.

      CustomResourceValidationExpressions Alpha Feature

      The CustomResourceValidationExpressions feature was introduced in 1.23. Once enabled, it allows the use of rules written in Common Expression Language (CEL) to validate CRDs.

      ServerSideFieldValidation Alpha Feature

      The ServerSideFieldValidation feature is introduced in 1.23. Once enabled, the apiserver can return warning messages for unknown or duplicate fields. The fieldValidation parameter can be specified in the request:
      Ignore: Before 1.23 and when the default behavior when the ServerSideFieldValidation feature is enabled, unknown or duplicate fields (except for the last one of the duplicate fields) will be ignored.
      Warn: The default behavior when the ServerSideFieldValidation feature is enabled.
      Strict: Request fails, returning an Invalid Request error.

      OpenAPIV3 feature enters the Beta phase

      The OpenAPIV3 feature introduced in 1.23 enters Beta in 1.24, allowing users to access the OpenAPI v3.0 specifications for all k8s types:
      $cluster/openapi/v3/apis/<group>/<version> : For a specific type
      $cluster/openapi/v3 : All types

      PodAndContainerStatsFromCRI Alpha Feature

      The PodAndContainerStatsFromCRI Alpha feature is introduced in 1.23. Once enabled, Pod metrics will be obtained via the CRI interface, instead of from cAdvisor.

      Other Updates

      Features Entering GA

      1.23:TTLAfterFinished,CSIVolumeFSGroupPolicy,IngressClassNamespacedParams,
      1.24:DefaultPodTopologySpread,PodOverhead,IndexedJob,PodAffinityNamespaceSelector,SuspendJob,CSRDuration,ServiceLBNodePortControl,PreferNominatedNode,ControllerManagerLeaderMigration(For more information, refer toMigrate Replicated Control Plane To Use Cloud Controller Manager)

      Upgraded to Beta Features

      1.23:JobTrackingWithFinalizers,StatefulSetMinReadySeconds.
      1.24:AnyVolumeDataSource,MixedProtocolLBService,GracefulNodeShutdownBasedOnPodPriority.

      Deprecation and Removal

      Parameter

      kube-apiserver:1.24 deprecates --master-count parameter --endpoint-reconciler-type=master-count option, by default using lease objects created by apiserver to synchronize the apiserver service's endpoint; 1.24 removes the following parameters:--address, --insecure-bind-address, --port,--insecure-port
      1.24 removes the following kube-controller-manager parameters:--address,--port,--deployment-controller-sync-period.
      kube-controller-manager and kube-scheduler's --port and --address parameters, which were deprecated in 1.23 and no longer work, were in 1.24.
      The legacy scheduler configuration methods and related parameters policy-config-file, policy-configmap, policy-configmap-namespace, and use-legacy-policy-config were removed in 1.23. You must use the KubeSchedulerConfiguration configuration file.
      1.23 removes the kubelet experimental-bootstrap-kubeconfig parameter, replacing it with the --bootstrap-kubeconfig parameter.
      1.23 removes the kubelet --seccomp-profile-root parameter and the seccompProfileRoot configuration item.
      1.24 deprecates the kubelet --pod-infra-container-image parameter. The sandbox image specified by this parameter will not be reclaimed. Future image reclamation processes will obtain sandbox image information from the CRI interface.

      API Version

      1.23
      rbac.authorization.k8s.io/v1alpha1 and scheduling.k8s.io/v1alpha1 have been removed, use bac.authorization.k8s.io/v1 and scheduling.k8s.io/v1 instead.
      The v1beta1 version of scheduler configuration has been removed, please use the v1beta2 or v1beta3 configuration file.
      1.24
      Remove client.authentication.k8s.io/v1alpha1 ExecCredential, and use client.authentication.k8s.io/v1 instead.
      Remove node.k8s.io/v1alpha1 RuntimeClass, and use node.k8s.io/v1 instead.
      Remove audit.k8s.io/v1alpha1, audit.k8s.io/v1beta1, and use audit.k8s.io/v1 instead.
      Deprecate CSIStorageCapacity storage.k8s.io/v1beta1 version, and use storage.k8s.io/v1 instead.

      Others

      1.24 removes the Service tolerate-unready-endpoints annotation deprecated since 1.11, and use Service.spec.publishNotReadyAddresses instead.
      1.24 removes deprecated features: ValidateProxyRedirects, StreamingProxyRedirects.
      metadata.clusterName field is deprecated in 1.24. This read-only field is always empty, leading to misunderstandings.

      Changelogs

      

      1.22 Changes Since 1.20

      Major Updates

      PodSecurityPolicy deprecated

      PodSecurityPolicy is deprecated in 1.21 and will be removed in 1.25. You can evaluate and migrate it to Pod Security Admission or third-party admission plug-ins.

      Immutable Secret and ConfigMap GA

      After Secret and ConfigMap are set as immutable (immutable: true), kubelet no longer watches the changes of these objects and mounts them to the container again to reduce the load of apiserver. This feature enters GA in 1.21.

      CronJobs GA

      CronJobs enters GA (batch/v1) in 1.21, and the new version controller CronJobControllerV2 with higher performance is enabled by default.

      IPv4/IPv6 supports to enter Beta

      Dual-stack networks allow Pods, services, and nodes to obtain IPv4 and IPv6 addresses. In 1.21, the dual-stack network is upgraded from alpha to beta and enabled by default.

      Graceful Node Shutdown

      This feature enters the beta phase in 1.21, allowing for notifying kubelet upon node shutdown events and gracefully terminates Pods on nodes.

      Persistent Volume Health Monitoring

      This alpha feature is introduced in 1.21, allowing for monitoring the running status of PV and marking when they become unhealthy. At this time, workloads can be adjusted accordingly to avoid data being written to or read from unhealthy PVs.

      Server-side Apply GA

      Server-side Apply helps users and controllers manage resources through declarative configuration, such as creating or modifying objects declaratively. Server-side Apply enteres the GA phase in 1.22.

      External Credential GA

      External credentials enter the GA phase in 1.22, providing better support for interactive login process plug-ins. For more information, refer to sample-exec-plugin.

      Etcd Updated to 3.5.0

      The Etcd 3.5.0 version is used by default in 1.22, which has improved security, performance, monitoring and developer experience, fixed multiple bugs, and added important new features such as structured log records and built-in log rotation.

      MemoryQoS

      The alpha MemoryQoS feature is supported starting from 1.22. Once enabled, the Cgroups v2 API will be used to manage and control memory allocation and isolation, ensuring memory usage for workloads and improving the availability of workloads and nodes in the case of memory resource competition. This feature was proposed by Tencent Cloud and contributed to the community.

      Cluster's Default seccomp Configuration

      Kubelet introduces the SeccompDefault alpha feature in 1.22. According to the --seccomp-default parameter and setting, kubelet will use the RuntimeDefault seccomp configuration instead of Unconfined to improve the security of workloads.

      Other Updates

      GA Features:
      1.21: EndpointSlice,Sysctls,PodDisruptionBudget.
      1.22: CSIServiceAccountToken
      Features Entering to beta:
      1.21: TTLAfterFinished
      1.22: SuspendJob,PodDeletionCost,NetworkPolicyEndPort.
      The new scheduler scoring plug-in NodeResourcesFit is introduced in 1.22, which is used to replace three plug-ins: NodeResourcesLeastAllocated, NodeResourcesMostAllocated, and RequestedToCapacityRatio.
      After the alpha feature APIServerTracing is enabled since 1.22, the apiserver supports distributed tracing and allows users to use the --service-account-issuer parameter to set multiple issuers. In addition, it can provide uninterrupted service when issuers are changed.

      Deprecation and Removal

      Removed Parameters and Features

      1. Service TopologyKeys is deprecated and replaced with Topology Aware Hints.
      2. kube-proxy
      Starting from 1.21, net.ipv4.conf.all.route_localnet=1 will not be automatically set in ipvs mode. For upgraded nodes, net.ipv4.conf.all.route_localnet=1 will be retained. But for new nodes, the default system value (usually 0) is inherited.
      The --cleanup-ipvs parameter is deleted and can be replaced with the --cleanup parameter.
      3. kube-controller-manager
      Starting from 1.22, the --horizontal-pod-autoscaler-use-rest-clients parameter is removed.
      The --port and --address parameters become invalid and will be removed in 1.24.
      4. kube-scheduler: The --hard-pod-affinity-symmetric-weight and --scheduler-name parameters are removed in 1.22, and instead, these information can be configured in the config file.
      5. Kubelet: The DynamicKubeletConfig feature is deprecated and is disabled by default. If the --dynamic-config-dir parameter is set when kubelet is started, an alarm will be reported.

      Removed or Deprecated Versions

      1. CronJob batch/v2alpha1 is removed started from 1.21
      2. Starting from 1.22, the following beta APIs are removed: (For more information, refer to Kubernetes official documentation.)
      rbac.authorization.k8s.io/v1beta1
      admissionregistration.k8s.io/v1beta1
      apiextensions.k8s.io/v1beta1
      apiregistration.k8s.io/v1beta1
      authentication.k8s.io/v1beta1
      authorization.k8s.io/v1beta1
      certificates.k8s.io/v1beta1
      coordination.k8s.io/v1beta1
      extensions/v1beta1 and networking.k8s.io/v1beta1 ingress API

      Change logs

      kubernetes1.22changelog
      kubernetes1.21changelog

      1.20 Changes Since 1.18

      Major Updates

      New Version of CronJob Controller

      The new version of the CronJob controller is introduced in 1.20, which uses the informer mechanism to replace the original polling and optimize the performance. You can set --feature-gates="CronJobControllerV2=true" in kube-controller -manager to enable the new version. The new version will be enabled by default on later versions.

      Dockershim Deprecation

      Dockershim has been deprecated. The kubernetes' support for Docker is deprecated and will be removed from a future release. OCI-compliant images generated by Docker will continue to run in the CRI-compliant runtime. For more information, refer to Don't Panic: Kubernetes and Docker and Dockershim Deprecation FAQ.

      Structured Logs

      The structures of log messages and k8s object reference are standardized to make log parsing, processing, storage, query, and analysis easier. Two methods are added to klog to support structured logs: InfoS and ErrorS. The --logging-format parameter is added to all components, and its default value is text in the previous format. You can set it to json to support structured logs, and the following parameters will become invalid: --add_dir_header, --alsologtostderr, --log_backtrace_at, --log_dir, --log_file, --log_file_max_size, --logtostderr, --skip_headers, --skip_log_headers, --stderrthreshold, --vmodule, and --log-flush-frequency.

      Exec Probe Timeout Processing

      A longstanding bug regarding exec probe timeouts that may impact existing Pod definitions has been fixed. Prior to this fix, the timeoutSeconds field was not respected for exec probes. Instead, probes would run indefinitely, even beyond their configured deadline, until a result was returned. With this change, the default value of 1 second will be applied if no value is specified. If a probe takes longer than 1 second, existing Pod definitions may need to be modifed, displaying the specified timeoutSeconds field. A switch called ExecProbeTimeout has also been added with this fix, allowing for retaining previous behaviors (In later releases, this feature will be locked and removed). To retain previous behaviors, it is required to set ExecProbeTimeout to false. For more information, refer to Configure Liveness, Readiness and Startup Probes.

      Volume Snapshot Operation Feature to GA

      This feature provides a standard way to trigger volume snapshot operations and allows you to incorporate snapshot operations in a portable manner on any Kubernetes environment and supported storage providers. Additionally, these Kubernetes snapshot primitives act as basic building blocks that unlock the ability to develop advanced and enterprise-grade storage administration features for Kubernetes, including application or cluster level backup solutions. Note that snapshot support requires Kubernetes distributors to bundle and deploy the snapshot controller, snapshot CRDs, and validation webhook, as well as a CSI driver supporting the snapshot feature.

      kubectl debug enters the beta phase

      The kubectl alpha debug command enters the beta phase, and is replaced with kubectl debug. It supports common debugging work directly from kubectl, for example:
      Troubleshoot workloads that crash on startup by creating a copy of the Pod with a different container image or command.
      Troubleshoot distroless and other containers that do not contain debugging tools by adding a new container with debugging tools, either in a new copy of the Pod or using an ephemeral container. (EphemeralContainers are an alpha feature that are not enabled by default.)
      Troubleshoot on a node by creating a container running in the host namespaces and with access to the host's file system. Note that as a new built-in command, kubectl debug takes priority over any kubectl plugin named debug, and the affected plugins must be renamed. kubectl alpha debug is now deprecated and will be removed in a subsequent version, and it is required to be updated to kubectl debug. For more information, refer to Debug Running Pods.

      API Priority and Fairness enters the beta phase

      The API Priority and Fairness feature introduced in 1.18 will be enabled by default in 1.20, allowing kube-apiserver to categorize incoming requests by priority.

      PID Limit Feature GA

      SupportNodePidsLimit (node-to-Pod PID isolation) and SupportPodPidsLimit (ability to limit PIDs per Pod) have both moved to the GA phase.

      Alpha Feature: Graceful Node Shutdown

      Users and cluster admins expect that Pods will adhere to the expected Pod lifecycle, including Pod termination. Currently, when a node shuts down, Pods do not follow the expected Pod termination lifecycle and are not terminated normally, which may cause issues for some workloads. The GracefulNodeShutdown feature is now added for alpha in 1.20, making the kubelet be able to monitor the system shutdown events of nodes, thereby enabling graceful termination of Pods during a system shutdown.

      CSIVolumeFSGroupPolicy enters the beta phase

      CSIDrivers can use the fsGroupPolicy field to control whether ownership and permissions can be controlled upon mount. (ReadWriteOnceWithFSType, File, and None)

      ConfigurableFSGroupPolicy enters the beta phase

      The following can be set in a non-recursive manner: fsgroup - PodFSGroupChangePolicy = OnRootMismatch

      Other Updates

      The Cloud Controller Manager component is added.
      Features graduating to GA:
      RuntimeClassnode.k8s.io/v1beta1 is deprecated and replaced with node.k8s.io/v1.
      Built-in API types defaults
      StartupProbe
      Services and Endpoints support AppProtocol field
      TokenRequest and TokenRequestProjection
      SCTPSupport
      Containerd support for Windows
      Ingress networking.k8s.io/v1beta1 is deprecated (it will be removed in 1.22) and replaced by networking.k8s.io/v1.
      seccomp seccomp annotations seccomp.security.alpha.kubernetes.io/pod and container.seccomp.security.alpha.kubernetes.io/... are deprecated (they will be removed in 1.22). You can directly specify the following fields for Pods and container specs:
      securityContext:
      seccompProfile:
      type: RuntimeDefault|Localhost|Unconfined ## choose one of the three
      localhostProfile: my-profiles/profile-allow.json ## only necessary if type == Localhost
      K8S converts annotations and fields automatically without additional operation.
      Node Certificate Automatic Rotation
      Limit Node Access to API Node authentication mode features are all implemented.
      Redesign Event API To reduce the impact of events on the system performance and add more fields to provide more useful information, Event API is redesigned. This work is done in 1.19.
      CertificateSigningRequest API In addition to certificates.k8s.io/v1beta1, the certificates.k8s.io/v1 version is added to CertificateSigningRequest. When using certificates.k8s.io/v1:
      You must specify spec.signerName and stop using kubernetes.io/legacy-unknown.
      You must specify spec.usages, which can not contain repeated value but only known usage.
      You must specify status.conditions[*].status.
      status.certificate must be PEM encoded and can contain only the CERTIFICATE block.
      Enter the Beta feature: The following features enter the beta phase and are enabled by default:
      EndpointSliceProxying kube-proxy reads information from EndpointSlices instead of Endpoints, which can greatly improve the cluster scalability and make it easier to add new features such as topology-aware routing.
      KubeSchedulerConfiguration
      HugePageStorageMediumSize
      ImmutableEphemeralVolumes The Secret and ConfigMap volumes can be marked as immutable. When there are many Secret and ConfigMap volumes, the pressure on apiserver can be greatly mitigated.
      NodeDisruptionExclusion
      NonPreemptingPriority
      ServiceNodeExclusion
      RootCAConfigMap
      Pod Resource Metrics in Scheduler
      ServiceAccountIssuerDiscovery

      Deprecations and Removals

      Deprecated Version

      Deprecated Version
      New Version
      apiextensions.k8s.io/v1beta1
      apiextensions.k8s.io/v1
      apiregistration.k8s.io/v1beta1
      apiregistration.k8s.io/v1
      authentication.k8s.io/v1beta1
      authentication.k8s.io/v1
      authorization.k8s.io/v1beta1
      authorization.k8s.io/v1
      autoscaling/v2beta1
      autoscaling/v2beta2
      coordination.k8s.io/v1beta1
      coordination.k8s.io/v1
      storage.k8s.io/v1beta1
      storage.k8s.io/v1

      kube-apiserver

      1. The componentstatus API is deprecated. This API is to provide the running status of etcd, kube-scheduler and kube-controller-manager components, but only worked when those components were local to apiserver, and when kube-scheduler and kube-controller-manager exposed unsecured health endpoints. After this API is deprecated, the etcd health check is included in the kube-apiserver health check and kube-scheduler/kube-controller-manager health checks can be made directly against those components' health endpoints.
      2. The apiserver no longer listens on insecure ports. The --address and --insecure-bind-address parameters can be set, but are invalid. The --port and --insecure-port parameters can be set to 0 only. These parameters will be removed in 1.24.
      3. TokenRequest and TokenRequestProjection enter the GA phase. You need to set the following parameters for kube-apiserver:
      --service-account-issuer: Identify the fixed URL of the cluster API server.
      --service-account-key-file: One or multiple public keys for token verification.
      --service-account-signing-key-file: Private key for issuing service account, which can use the same file as the --service-account-private-key-file parameter of kube-controller-manager.

      kubelet

      1. The following parameters are removed:
      --seccomp-profile-root
      --cloud-provider and --cloud-config, which are replaced with config
      --really-crash-for-testing and --chaos-chance
      2. The deprecated metrics/resource/v1alpha1 endpoint is removed and please replace with metrics/resource.

      Other Removals

      The failure-domain.beta.kubernetes.io/zone and failure-domain.beta.kubernetes.io/region labels are deprecated and please replace with topology.kubernetes.io/zone and topology.kubernetes.io/region respectively. All labels prefixed with failure-domain.beta... labels should be replaced with the labels prefixed with corresponding topology....
      PodPreset is removed, and you can use webhooks to implement this feature.
      The basic auth authentication method is no longer supported.
      Direct CBS inline mounting to workloads is no longer supported.
      Note
      When you upgrade from Kubernetes 1.18 to 1.20, successful mounting of CSI ephemeral (inline) volumes cannot be guaranteed. If your application uses a CSI ephemeral volume, we recommend you convert it to a persistent volume before upgrade.

      Change logs

      1.18 Changes Since 1.16

      Major Updates

      Cloud Provider labels reach to the stable (GA) phase

      Deprecated and new labels are as listed below:
      Deprecated Labels
      New Labels
      beta.kubernetes.io/instance-type
      node.kubernetes.io/instance-type
      failure-domain.beta.kubernetes.io/region
      topology.kubernetes.io/region
      failure-domain.beta.kubernetes.io/zone
      topology.kubernetes.io/zone

      Volume Snapshot enters the Beta phase

      VolumeSnapshotDataSource is enabled by default. For more information, see Kubernetes 1.17 Feature: Kubernetes Volume Snapshot Moves to Beta.

      CSI Migration enters the Beta phase

      CSI Migration is enabled by default. For more information, see Kubernetes 1.17 Feature: Kubernetes In-Tree to CSI Volume Migration Moves to Beta.

      Graduation of Kubernetes Topology Manager to Beta

      The TopologyManager feature enters beta in 1.18. This feature enables NUMA alignment of CPU and devices (such as SR-IOV VFs) that will allow your workload to run in an environment optimized for low latency. Prior to the introduction of the TopologyManager, the CPU and Device Manager would make resource allocation decisions independent of each other. This could result in undesirable allocations on multi-socket CPU systems, causing degraded performance on latency critical applications.

      Graduation of Server-Side Apply to Beta 2

      Server-Side Apply was upgraded to beta on Kubernetes 1.16, but is now introducing a second Beta (ServerSideApply) on Kubernetes 1.18. This new version will track and manage changes to fields of all new Kubernetes objects, allowing uses to know resource changes.

      IngressClass Resources

      The IngressClass resource is used to describe a type of Ingress controller within a Kubernetes cluster. Ingresses uses the new ingressClassName field to set up the controller name for IngressClass, and replaces the depreciated kubernetes.io/ingress.class annotation.

      Other Updates

      Graduation of NodeLocal DNSCache to GA.
      Graduation of IPv6 to Beta.
      kubectl debug: Alpha Feature.
      Windows CSI support: Alpha Feature.
      ImmutableEphemeralVolumes: Alpha Feature (supporting immutable ConfigMaps and Secrets without refreshing the corresponding volumes).
      The following features graduate to GA:
      ScheduleDaemonSetPods
      TaintNodesByCondition
      WatchBookmark
      NodeLease
      CSINodeInfo
      VolumeSubpathEnvExpansion
      AttachVolumeLimit
      ResourceQuotaScopeSelectors
      VolumePVCDataSource
      TaintBasedEvictions
      BlockVolume, CSIBlockVolume
      Windows RunAsUserName
      Features graduating to Beta:
      EndpointSlices: Disabled by default
      CSIMigrationAWS: Disabled by default
      StartupProbe
      EvenPodsSpread

      Deprecations and Removals

      Removed Features

      The following features, which are enabled by default and cannot be configured, are removed:
      GCERegionalPersistentDisk
      EnableAggregatedDiscoveryTimeout
      PersistentLocalVolumes
      CustomResourceValidation
      CustomResourceSubresources
      CustomResourceWebhookConversion
      CustomResourcePublishOpenAPI
      CustomResourceDefaulting

      Other Removals

      The following built-in cluster roles are removed:
      system:csi-external-provisioner
      system:csi-external-attacher

      Deprecated Feature Switches and Parameters

      The default service IP CIDR block ( 10.0.0.0/24) is deprecated, and must be set through the --service-cluster-ip-range parameter on kube-apiserver.
      The rbac.authorization.k8s.io/v1alpha1 and rbac.authorization.k8s.io/v1beta1 API groups are deprecated and will be removed in 1.20. Therefore, migrate your resources to rbac.authorization.k8s.io/v1.
      The CSINodeInfo feature gate is deprecated. This feature has graduated to GA and is enabled by default.

      Parameter and Other Changes

      kube-apiserver

      --encryption-provider-config: If cacheSize: 0 is specified in the configuration file, versions earlier than 1.18 are automatically configured to cache 1,000 keys, while version 1.18 will report a configuration verification error. You can disable the cache by setting cacheSize to a negative value.
      --feature-gates: The following features are enabled by default and can no longer be configured through the command line.
      PodPriority
      TaintNodesByCondition
      ResourceQuotaScopeSelectors
      ScheduleDaemonSetPods
      The following resource versions (group versions) are no longer supported:
      apps/v1beta1 and apps/v1beta2, which are replaced with apps/v1.
      Under extensions/v1beta1:
      daemonsets, deployments and replicasets, which are replaced with apps/v1.
      networkpolicies, which is replaced with networking.k8s.io/v1.
      podsecuritypolicies, which is replaced with policy/v1beta1.

      kubelet

      --enable-cadvisor-endpoints: This parameter is disabled by default. To access the cAdvisor v1 JSON API, you must enable it.
      The --redirect-container-streaming parameter is deprecated and will be removed from later versions. 1.18 only supports the default behavior (kubelet proxy for streaming requests). If --redirect-container-streaming=true is set, it must be removed.
      The /metrics/resource/v1alpha1 endpoint is deprecated and replaced with /metrics/resource.

      kube-proxy

      The following parameters are deprecated:
      The --healthz-port is deprecated and replaced with --healthz-bind-address.
      The --metrics-port is deprecated and replaced with --metrics-bind-address.
      The EndpointSliceProxying feature gate (disabled by default) is added to control whether to enable EndpointSlices in kube-proxy. The EndpointSlice feature gate no longer affects the behaviors of kube-proxy.
      The following timeout settings for IPVS connection configuration are added:
      --ipvs-tcp-timeout
      --ipvs-tcpfin-timeout
      --ipvs-udp-timeout
      The iptables mode supports the IPv4/IPv6 dual-protocol stack.

      kube-scheduler

      The scheduling_duration_seconds metric is deprecated:
      The scheduling_algorithm_predicate_evaluation_seconds is deprecated and replaced with framework_extension_point_duration_seconds[extension_point="Filter"]
      The scheduling_algorithm_priority_evaluation_seconds is deprecated and replaced with framework_extension_point_duration_seconds[extension_point="Score"]
      The AlwaysCheckAllPredicates is deprecated in the scheduler policy API.

      -enable-profiling Parameter

      To align kube-apiserver, kube-controller-manager and kube-scheduler, profiling is enabled by default. To disable profiling, it is required to specify the --enable-profiling=false parameter.

      kubectl

      The deprecated --include-uninitialized parameter is removed.
      The kubectl and k8s.io/client-go no longer use http://localhost:8080 as the default apiserver address.
      The kubectl run supports Pod creation and no longer supports using the deprecated generator to create other types of resources.
      The deprecated kubectl rolling-update command is removed and please use the rollout command.
      The –dry-run supports three parameter values: client, server, and none.
      The –dry-run=server supports the following commands: apply, patch, create, run, annotate, label, set, autoscale, drain, rollout undo, and expose.
      The kubectl alpha debug command is added, which can be used for debugging and troubleshooting on ephemeral containers in Pods (the EphemeralContainers feature introduced on version 1.16 needs to be enabled).

      hyperkube

      The implementation of hyperkube is changed from Go code to a bash script.

      Change Logs

      1.16 Changes Since 1.14

      Major updates

      Improved Cluster Stability and Availability

      Production-ready features like bare metal cluster tool and high availability (HA) have been improved and enhanced. kubeadm support for HA capability enters the beta phase, allowing you to use the kubeadm init and kubeadm join commands to configure and deploy the highly available HA control plane. Certificate management has become more robust, with kubeadm now seamlessly updating all your certificates before they expire. For more information, see pr357 and pr970.

      Continuous CSI Improvement

      SIG Storage continues work to enable migration of in-tree volume plugins to Container Storage Interface (CSI). It works on bringing CSI to feature parity with in-tree functionality, including resizing and inline volumes. It introduces new alpha functionality in CSI that doesn't exist in the Kubernetes Storage subsystem yet, like volume cloning. Volume cloning enables you to specify another PVC as a DataSource when configuring a new volume. If the underlying storage system supports this functionality and implements the CLONE_VOLUME capability in its CSI driver, the new volume becomes a clone of the source volume. For more information, see pr625.

      Features

      Features Graduating to GA:
      CRD
      Admission Webhook
      GCERegionalPersistentDisk
      CustomResourcePublishOpenAPI
      CustomResourceSubresources
      CustomResourceValidation
      CustomResourceWebhookConversion
      CSI support for volume resizing graduates to Beta.

      General Updates

      Go modules are supported by the Kubernetes core.
      Preparation on cloud provider code extraction and organization is continued. The cloud provider code has been moved to kubernetes/legacy-cloud-providers for easier removal later and external usage.
      kubectl get and describe command support extension
      Nodes support 3rd party device monitoring plugins.
      A new alpha version of scheduling framework for developing and managing plugins and extending the scheduler features is introduced. For more information, see pr624.
      extensions/v1beta1, apps/v1beta1 and apps/v1beta2 APIs continue to be depreciated. These extensions will be retired in 1.16.
      The Topology Manager component is added to Kubelet, aiming to coordinate resource allocation decisions and optimize resource allocation.
      IPv4/IPv6 dual stack is supported to assign both v4 and v6 addresses to Pods and Services.
      The API Server Network Proxy in alpha Feature.
      More extension options are provided for cloud controller manager migration.
      extensions/v1beta1, apps/v1beta1 and apps/v1beta2 APIs are deprecated.

      Known Issues

      Using the --log-file parameter is known to be problematic on Kubernetes 1.15. This presents as things being logged multiple times to the same file. For more information, see [Failing Test] timeouts in ci-kubernetes-e2e-gce-scale-performance.

      Update notes

      Cluster
      The following labels can no longer be added to new nodes: beta.kubernetes.io/metadata-proxy-ready, beta.kubernetes.io/metadata-proxy-ready and beta.kubernetes.io/kube-proxy-ds-ready.
      The ip-mask-agent uses node.kubernetes.io/masq-agent-ds-ready as the node selector and no longer uses beta.kubernetes.io/masq-agent-ds-ready.
      The kube-proxy uses node.kubernetes.io/kube-proxy-ds-ready as the node selector and no longer uses beta.kubernetes.io/kube-proxy-ds-ready.
      The metadata-proxy uses cloud.google.com/metadata-proxy-ready as the node selector and no longer uses beta.kubernetes.io/metadata-proxy-ready.
      API Machinery k8s.io/kubernetes and other published components (such as k8s.io/client-go and k8s.io/api) now contain Go module files, including version information of the dependent library. For more information on consuming k8s.io/client-go in Go modules, see go-modules and pr74877.
      Apps: Hyperkube short aliases have been removed from source code, because these aliases will be created when compiling hyperkube docker image. For more information, see pr76953.
      Lifecycle
      Deprecated kubeadm v1alpha3 configurations have been totally removed.
      kube-up.sh no longer supports centos and local.
      Storage
      The Node.Status.Volumes.Attached.DevicePath field is no longer set for CSI volumes. You must update any external controllers that depend on this field.
      CRDs in alpha version are removed.
      The StorageObjectInUseProtection admission plugin is enabled by default. If you previously had not enabled it, your cluster behavior may change.
      When PodInfoOnMount is enabled for a CSI driver, the new csi.storage.k8s.io/ephemeral parameter in the volume context allows a driver's NodePublishVolume implementation to determine on a case-by-case basis whether the volume is ephemeral or a normal persistent volume. For more information, see pr79983.
      The VolumePVCDataSource (Storage Volume Cloning feature) enters beta. For more information, see pr81792.
      Limits for in-tree and CSI volumes are integrated into one scheduler predicate. For more information, see pr77595.
      kube-apiserver
      The --enable-logs-handler parameter is deprecated and will be removed in v1.19.
      The --basic-auth-file flag and authentication mode are deprecated and will be removed from a future release.
      The default service IP CIDR block ( 10.0.0.0/24) is deprecated and will be removed in six months/two releases. The --service-cluster-ip-range parameter is required to configure the service IP range.
      kube-scheduler The v1beta1 Event API is used. Any tool targeting scheduler events needs to use it.
      kube-proxy
      The --conntrack-max parameter is removed and replaced with --conntrack-min and --conntrack-max-per-core.
      The --cleanup-iptables parameter is removed.
      The --resource-container is removed.
      kubelet
      The --allow-privileged, --host-ipc-sources, --host-pid-sources and --host-network-sources parameters are removed and replaced with the admission controller of PodSecurityPolicy.
      The cAdvisor JSON API is deprecated.
      The --containerized is removed.
      The --node-labels parameter can no longer be used to configure forbidden labels prefixed with kubernetes.io- or k8s.io-.
      kubectl
      The kubectl scale job is removed.
      The --pod/-p parameter of the kubectl exec command is removed.
      The kubectl convert command is removed.
      The --include-uninitialized is removed.
      The kubectl cp no longer supports copying symbolic links from containers. You can use the following commands instead:
      local to pod: tar cf - /tmp/foo | kubectl exec -i -n <some-namespace> <some-pod> -- tar xf - -C /tmp/bar
      pod to local: kubectl exec -n <some-namespace> <some-pod> -- tar cf - /tmp/foo | tar xf - -C /tmp/bar
      kubeadm
      The kubeadm upgrade node config and kubeadm upgrade node experimental-control-plane commands are deprecated and replaced with kubeadm upgrade node.
      The --experimental-control-plane parameter is deprecated and replaced with --control-plane.
      The --experimental-upload-certs parameter is deprecated and replaced with --upload-certs.
      The kubeadm config upload command is deprecated and replaced with kubeadm init phase upload-config.
      CoreDNS checks readiness via the ready plugin.
      The proxy plugin is deprecated and replaced with the forward plugin.
      The resyncperiod option is removed from the kubernetes plugin.
      The upstream option is deprecated. If it is specified, it will be ignored.

      Change Logs

      1.14 Changes Since 1.12

      Major Updates

      Bump CSI Spec to 1.0.0 and gRPC to 1.13.0.
      Make CoreDNS default in kubeup and update CoreDNS version/manifest in kubeup and kubeadm.
      kubeadm is used to simplify cluster management.
      Support Windows Nodes to enter stable.
      Durable (non-shared) local storage management.
      Support total process ID limiting for nodes.
      Support Pod Priority and Preemption.

      General Updates

      dry-run graduates to beta (dry-run enables you to simulate real API requests without actually changing the cluster status).
      kubectl diff graduates to beta.
      kubectl plugin registration becomes stable.
      kubelet plugin mechanism graduates to beta.
      CSIPersistentVolume graduates to GA.
      TaintBasedEviction graduates to beta.
      kube-scheduler perception of volume topology becomes stable.
      Support for out-of-tree CSI volume plugins becomes stable.
      Support third-party device monitoring plugins.
      The kube-scheduler subnet feasibility graduates to beta.
      Pod Ready supports customizing probe conditions.
      Node memory supports HugePage.
      The RuntimeClass graduates to beta.
      Node OS/Arch labels graduate to GA.
      Node-leases graduate to beta.
      The kubelet resource metrics endpoint graduates to alpha and supports data collection through Prometheus.
      The runAsGroup graduates to beta.
      The kubectl apply server-side graduates to alpha, allowing you to perform apply operations on the server side.
      The kubectl supports kustomize.
      The resolv.conf can be configured in Pods.
      CSI volumes support resizing.
      CSI supports topology.
      Volume mounting supports configuration of sub-path parameters.
      CSI supports raw block devices.
      CSI supports local ephemeral volumes.

      Update Notes

      kube-apiserver
      The etcd2 is no longer supported. The --storage-backend=etcd3 is used by default.
      The --etcd-quorum-read parameter is deprecated.
      The --storage-versions parameter is deprecated.
      The --repair-malformed-updates parameter is deprecated.
      kube-controller-manager The --insecure-experimental-approve-all-kubelet-csrs-for-group parameter is deprecated.
      kubelet
      The --google-json-key parameter is deprecated.
      The --experimental-fail-swap-on parameter is deprecated.
      kube-scheduler componentconfig/v1alpha1 is no longer supported.
      kubectl The run-container command is no longer supported.
      taints The node.alpha.kubernetes.io/notReady and node.alpha.kubernetes.io/unreachable are no longer supported and replaced with node.kubernetes.io/not-ready and node.kubernetes.io/unreachable respectively.

      Change Logs

      1.12 Changes Since 1.10

      Major Updates

      API

      Subresources for CustomResources graduate to beta now and are enabled by default. With this, updates to the /status subresource will disallow updates to all fields other than .status (not just .spec and .metadata as before). Also, required and description can be used at the root of the CRD OpenAPI validation schema when the /status subresource is enabled. In addition, you can now create multiple versions of CustomResourceDefinitions, but without any kind of automatic conversion, and CustomResourceDefinitions now allow specification of additional columns for kubectl get output via the spec.additionalPrinterColumns field.
      The dry run feature is supported. It allows you to view the execution results of some commands without having to submit relevant modifications.

      Authentication and Authorization

      RBAC aggregation of ClusterRoles graduates to GA. The client-go credentials plugin graduates to beta, allowing you to get TLS authentication information from external plugins.
      The following annotations are added to audit events, so that you can be better informed of the audit decision-making process:
      The Authorization component sets authorization.k8s.io/decision (the allow or forbid authorization decision) and authorization.k8s.io/reason (the reason for this decision).
      The PodSecurityPolicy admission controller sets podsecuritypolicy.admission.k8s.io/admit-policy and podsecuritypolicy.admission.k8s.io/validate-policy annotations containing the name of the policy that allows a Pod to be admitted. ( PodSecurityPolicy also gains the ability to limit hostPath volume mounts to be read-only.)
      The NodeRestriction admission controller prevents nodes from modifying taints on their node objects, making it easier to keep track of which nodes should be in use.

      CLI Command Line

      CLI implements a new plugin mechanism, providing a library with common CLI tooling for plugin authors and further refactorings of the code.

      Internet

      The IPVS mode graduates to GA.
      CoreDNS graduates to GA to replace kube-dns.

      Node

      DynamicKubeletConfig graduates to the Beta phase.
      cri-tools graduates to GA.
      PodShareProcessNamespace graduates to the Beta phase.
      Alpha features RuntimeClass and CustomCFSQuotaPeriod are added.

      Scheduler

      Pod Priority and Preemption graduate to the Beta phase.
      DaemonSet Pod scheduling is no longer managed by the DaemonSet controller, but by the default scheduler.
      TaintNodeByCondition graduates to the Beta phase.
      The Use Local Image First feature is enabled by default. During Pod scheduling, nodes that have locally pulled the images required by all or some Pods will have a higher priority. This accelerates the launch of Pods.

      General Updates

      Features graduating to GA: ClusterRole and StorageObjectInUseProtection.
      Features graduating to Beta: External Cloud Provider.

      Update Notes

      kube-apiserver
      The --storage-version parameter is removed and replaced with --storage-versions. The --storage-versions parameter is also deprecated.
      The default value of --endpoint-reconciler-type is changed to lease.
      When --enable-admission-plugins is used, it is contained by default. When the --admission-control parameter is used, it must be explicitly specified.
      kubelet
      The --rotate-certificates parameter is deprecated and replaced with the .RotateCertificates field in the configuration file.
      kubectl
      All kubectl run generators except run-pod/v1 are deprecated.
      The --interactive parameter is removed from kubectl logs.
      The --use-openapi-print-columns is deprecated and replaced with --server-print.

      Change Logs

      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy