As the world of container orchestration continues to evolve, Kubernetes 1.28 emerges as a testament to relentless progress and innovation. With features ranging from the revamped kubectl events command to sophisticated dynamic resource allocation, this version promises to redefine the way we perceive and manage container ecosystems.
Whether you're a cluster admin interested in monitoring metrics or a developer eyeing performance enhancements, Kubernetes 1.28 holds a plethora of upgrades designed to improve your orchestration experience.
Dive in with us as we unpack the remarkable enhancements that will shape the next generation of container management.
Four Intriguing Enhancements in Kubernetes 1.28
- Sidecar Container Integration
The sidecar pattern has long been recognized as a pivotal practice in Kubernetes. With the 1.28 release, sidecar containers are officially supported, streamlining functions like metrics collection, secret management, and service mesh integration. This development promises more robust Kubernetes workloads, much to the delight of developers, admins, and end-users. - Revamping Kubernetes Jobs
This release introduces several notable improvements to Kubernetes jobs, especially beneficial for Machine Learning tasks. From ensuring sidecars don't hinder job completion to advanced features like retry mechanisms for pod failures and more sophisticated job control, Kubernetes is clearly poised to be a significant player in the ML space. - Enhancements for Rolling Upgrades
Upgrading system components without significant downtimes has always been a challenge. Kubernetes 1.28 introduces features to make these rolling upgrades smoother and more reliable. Notably, the new functionalities promise reduced connection drops during upgrades, ensuring minimal service disruptions and a more seamless transition. - Community-Driven Package Management
In a bid to foster greater community involvement, Kubernetes is shifting away from Google-centric infrastructures, opting for community-owned repositories for package distributions. This evolution underscores the project's maturity and hints at a future driven by diverse contributions.
Advanced API Features
- Consistent Reads From Cache: Kubernetes 1.28 brings an efficiency overhaul to API requests, particularly GET and LIST. The introduction of WatchProgressRequest optimizes performance by fetching information from etcd's watch cache.
- Data Streaming with Informers: Building on the previous feature, the API server can now stream data changes in real-time, reducing the memory overhead of repetitive LIST requests.
- Mixed Version Proxy: Addressing the challenges of cluster upgrades, the kube-apiserver now has the capability to delegate resource requests to the appropriate peer servers, ensuring seamless operations across different server versions.
- Improved Custom Resource Definitions (CRD) Handling: Kubernetes 1.28 simplifies the task of managing CRD schema changes. For instance, edits that don't interfere with new validation rules won't trigger unnecessary validation errors, making it more user-friendly.
- CRD Validation via Expression Language: The new release also allows for the definition of validation rules directly within custom resources, eliminating the need for additional webhooks. This enhancement has undergone improvements since its Beta phase in Kubernetes 1.25.
- Enhanced Admission Control with CEL: Kubernetes 1.28 adopts the Common Expression Language (CEL) for its admission controller, offering more precise validation mechanisms. For instance, admins can set up rules to deny object creations that breach certain criteria.
Application-Based Features
- Refined Job Handling: Jobs in Kubernetes can now be replaced only once they're fully terminated, catering to specific workload requirements. Admins can easily set this behavior using the podReplacementPolicy field.
- Fine-Tuned Job Retries: The release also introduces granular control over job retries. Admins can now define retry limits for specific indices within a job, ensuring more robust and fault-tolerant operations.
Job Timestamp Annotations Added in CronJobs
SIG: sig-apps
Status: Beta
Feature: CronJobCreationAnnotation (Enabled by default)
Now, the CronJob controller includes the expected timestamp for when a job will run:
batch.kubernetes.io/cronjob-scheduled-timestamp: "2016-05-19T03:00:00-07:00"
Adjustable Job Pod Failure Management
SIG: sig-apps
Status: Beta
Feature: JobPodFailurePolicy (Enabled by default)
Manage pod failures in jobs with the new podFailurePolicy. It lets you control retries without impacting the backoffLimit.
Pod Index Label Introduced for StatefulSets & Indexed Jobs
SIG: sig-apps
Status: Beta
Feature: PodIndexLabel (Enabled by default)
Jobs & StatefulSet controllers now support the pod index as a label.
KMS v2 Enhancements
SIG: sig-auth
Status: Beta
Feature: KMSv2 (Enabled by default)
KMS v2 aims to solve several challenges of its predecessor, like performance and manual key rotation. The updated version brings various improvements, including a revamped key hierarchy and better observability.
Reduction of Old Service Account Tokens
SIG: sig-auth
Status: Beta
Features: LegacyServiceAccountTokenTracking & LegacyServiceAccountTokenCleanUp (Both enabled by default)
From Kubernetes 1.22, service account tokens have transitioned from being auto-generated to being sourced from the TokenRequest API. The new changes aim to further minimize the older style tokens.
Know Your Authenticated Self with the Auth API
SIG: sig-auth
Status: Stable
Feature: APISelfSubjectReview (Enabled by default)
The new API reveals the authenticated identity, be it a user or a ServiceAccount. The info can be accessed with a CLI command: kubectl alpha auth whoami, assisting in diagnosing authentication issues.
Kubectl Delete Gets Interactive with New Flag
SIG: sig-cli
Status: Alpha
Using the -i or --interactive flag during kubectl delete provides a preview of objects slated for deletion, allowing user confirmation.
Kubectl Events Command Upgrade
SIG: sig-cli
Status: Stable
The revamped kubectl events command is designed to overcome the limitations of kubectl get events, offering improved watch controls and filtering. It's stable from Kubernetes 1.28.
Metrics Stability Enhancement
SIG: sig-instrumentation
Status: Beta
The new update introduces two stability levels, 'Internal' and 'Beta', for metrics. The 'Beta' metrics provide more reliability and forward compatibility.
Kube-proxy Boosts Ingress Connection Health Management
SIG: sig-network
Status: Alpha
Feature: KubeProxyDrainingTerminatingNodes (Disabled by default)
Kube-proxy now ensures connections terminate gracefully when a node is ending. There's also a new /livez path for more precise health checks.
Enhanced NodePort Ranges Allocation
SIG: sig-network
Status: Beta
Feature: ServiceNodePortStaticSubrange (Enabled by default)
This update allows better control over NodePort service, reserving the initial ports in service-node-port-range for static allocation.
Load Balancer Node Management Improved
SIG: sig-network
Status: Beta
Feature: StableLoadBalancerNodeSet (Enabled by default)
The way nodes are removed from the load balancer's node list is being refined. This minimizes undesirable chain reactions like immediate connection terminations and load balancer resyncs.
EndpointSlice Reconciler Relocated
SIG: sig-network
Status: Stable
The EndpointSlice reconciler logic is being exposed and moved to a new module for use in custom Endpoint controllers.
Optimized iptables-restore Functioning
SIG: sig-network
Status: Stable
Feature: MinimizeIPTablesRestore (Enabled by default)
Kube-proxy will see improved performance in iptables mode by modifying how iptables-restore functions, focusing only on changed rules.
Cleanup of IPTables Chain Ownership
SIG: sig-network
Status: Stable
Feature: IPTablesOwnershipCleanup (Enabled by default)
As a cleanup measure, unused iptables chains, previously created by components like kubelet and kube-proxy, are being removed.
DNS Configuration Broadened
SIG: sig-network
Status: Stable
Feature: ExpandedDNSConfig (Enabled by default)
The DNS configuration is now more flexible, accommodating more search paths and longer lists.
Enhanced Traffic Direction for Terminating Endpoints
SIG: sig-network
Status: Stable
Feature: ProxyTerminatingEndpoints (Enabled by default)
Kube-proxy will better handle traffic to endpoints that are ending. This is especially useful during rolling updates when endpoints might be in terminating states.
For a comprehensive understanding, visit the Kubernetes.io article titled 'Advancements in Kubernetes Traffic Engineering'.
Condition for Sandbox Initiation (KEP-3085)
SIG: sig-node
Status: Introduction to Alpha
Feature: PodReadyToInitiateContainersCondition (Default: false)
Kubernetes 1.28 has rolled out a new pod status called PodReadyToInitiateContainersCondition. This status illuminates when a Pod is initiating its sandbox and configuring its networking, assisting cluster admins and Kubernetes controllers.
Determining cgroup Driver via CRI
SIG: sig-node
Status: Introduction to Alpha
Feature: KubeletCgroupDriverDetectionFromCRI (Default: false)
The Kubernetes 1.28 update lets the kubelet intuitively determine the cgroup driver through the container runtime. This aims to mitigate diagnostic complications from configuration discrepancies.
Modular kubelet Config Directory
SIG: sig-node
Status: Introduction to Alpha
Feature: ModularIndexedJob (Default: false)
With the 1.28 release, Kubernetes facilitates the configuration of a kubelet directory. Using the new --config-dir argument, users can now input multiple .conf configuration files, streamlining their setup processes.
Sidecar Container Management
SIG: sig-node
Status: Introduction to Alpha
Feature: ManagedSidecarContainers (Default: false)
The new update advances the management of sidecar containers. Kubernetes now provides an improved method to identify and manage a sidecar container's lifecycle in correlation with primary containers.
CDI Device Inclusion in Device Plugin API
SIG: sig-node
Status: Introduction to Alpha
Feature: DevicePluginCDIDevicesIntegration (Default: false)
Kubernetes 1.28 unveils an enhancement allowing device plugin developers to seamlessly convey requests to the container runtimes, elevating device integration capabilities.
Evolving Dynamic Resource Allocation
SIG: sig-node
Status: Progression to Alpha
Feature: ProgressiveDynamicResourceAssignment (Default: false)
This feature, continuing its journey since its introduction in Kubernetes 1.26, enables pods to request specific resources beyond conventional CPU and memory needs, catering especially to environments requiring specialized resources.
User Namespace Integration in Pods
SIG: sig-node
Status: Progression to Alpha
Feature: StatelessUserNamespaceIntegrationInPods (Default: false)
The integration of user namespaces with Kubernetes 1.28 augments security by allowing pod processes to run with varied user IDs than those on the host, primarily available for Linux systems.
Enhanced Multi-Numa Alignment in Topology Manager
SIG: sig-node
Status: Progression to Beta
Feature: TopologyManagerAdvancedPolicyOptions (Default: true)
Kubernetes now provides a heightened awareness of system topology, focusing on allocating tasks to proximate CPU cores for heightened efficiency.
Memory Swap Support for Nodes
SIG: sig-node
Status: Progression to Beta
Feature: NodeMemorySwap (Default: true)
Supporting swap on Linux nodes, this feature in Kubernetes 1.28 is poised to prevent out-of-memory terminations, though with a cautionary note on potential latency.
Summary:
Kubernetes 1.28 marks another milestone in the pursuit of impeccable container orchestration. From significant improvements in kubectl commands and metrics stability to advanced networking features like refined ingress connection health management, this update is brimming with features that cater to both the experienced and the newcomers.
Enhanced node management, optimization of iptables, and features focused on user security demonstrate Kubernetes' commitment to addressing community feedback and ever-evolving challenges.
As we look forward to what the future holds, Kubernetes 1.28 undoubtedly sets a high bar for future iterations, pushing the boundaries of what's possible in the realm of container orchestration.
Check out our latest blogpost
