Why Kubernetes Resource Optimization Requires More Than a Single Fix
Kubernetes resource optimization is the ongoing practice of matching a cluster’s CPU and memory requests to what workloads actually consume, instead of what they were provisioned for at launch. Most platform teams default to one of two habits: setting requests high to be safe, or setting them low and hoping usage stays under the limit. The first habit burns budget on capacity nobody uses. The second invites Out Of Memory Killed (OOMKilled) errors and CPU throttling the moment traffic spikes.
Neither habit is a strategy. Each is a guess made once and left in place. Resource requests and limits drift out of sync with real usage within weeks of being set, because usage shifts with release cycles, seasonal traffic, and time-of-day patterns that a single sizing exercise can’t anticipate. Treating optimization as a one-time YAML edit is a large part of why so many clusters stay over-provisioned indefinitely.
This article covers what actually causes that drift, a four-layer model for closing it, and a 90-day rollout plan that gets teams from manual guesswork to continuous optimization without downtime.
Why Static Resource Requests Lead to Overprovisioning at Scale
Overprovisioning happens when a team sets CPU and memory requests based on a single moment of observed traffic, then never adjusts them again. Requests are what a pod reserves on a node regardless of use; limits are the ceiling it’s allowed to consume before Kubernetes throttles CPU or kills the process for exceeding memory. Actual usage sits somewhere below both, and the gap between requested and used resources is where cloud spend disappears.
The problem compounds because usage isn’t static. A checkout service that needs 2 vCPUs during a flash sale might need a fraction of that overnight. A batch job’s memory footprint changes with every data volume it processes. Release cycles introduce new code paths with different resource profiles, and nobody goes back to re-tune requests every time a deploy ships. The result is a cluster full of pods sized for a moment that already passed.
The cost adds up fast at scale. A platform team spending $150,000 to $250,000 a month on cloud compute typically wastes $60,000 to $100,000 of that on idle allocation, capacity reserved by pods but never actually used. That waste doesn’t show up as a single line item. It’s spread across hundreds of deployments, which is exactly why it survives quarter after quarter without anyone catching it.
The risk of correcting too far in the other direction is just as real. A team that panics over a cost report and slashes requests across the board without checking actual usage first often trades a spend problem for a reliability one. Pods that were comfortably sized suddenly hit their memory limit under normal load, OOMKilled errors start appearing in incident channels, and CPU throttling adds latency that shows up in customer-facing metrics before anyone connects it back to the resource change. Overprovisioning and under-provisioning are two sides of the same root cause: sizing decisions made from a snapshot instead of from continuous, current data.
Fixing this requires visibility into requests versus real usage across the whole cluster, not just the handful of services someone happens to be reviewing that week. It also requires that visibility to stay current, since a dashboard built from last month’s usage is really just a slower version of the same one-time snapshot problem. Zesty‘s approach to compute cost visibility tracks that gap continuously across every namespace, which is the first step before any rightsizing or automation can happen.
The Four-Layer Kubernetes Autoscaling Model
Kubernetes autoscaling isn’t one system, it’s four layers stacked on top of each other, each solving a different part of the resource-matching problem and each with its own blind spot. Understanding where one layer ends and the next begins is the difference between a rollout that holds and one that gradually regresses within a quarter.
Layer 1: Pod-level rightsizing. The Vertical Pod Autoscaler (VPA) adjusts a pod’s CPU and memory requests based on observed usage. It solves the sizing problem at the individual pod level, but it requires restarting the pod to apply a new request, which makes it disruptive for anything latency-sensitive, and it has no visibility into what’s happening on other nodes or clusters.
Layer 2: Horizontal scaling. The Horizontal Pod Autoscaler (HPA) and Kubernetes Event-Driven Autoscaling (KEDA) add or remove replicas based on metrics like CPU utilization or queue depth. Both are reactive: they wait for a threshold to be crossed before acting, which means there’s always a lag between demand arriving and capacity catching up.
Layer 3: Node-level provisioning. Karpenter and cluster autoscaler launch and terminate nodes to match the pods that need to run. Node consolidation, packing pods efficiently onto fewer nodes, is where these tools focus, but they optimize for tight packing rather than leaving headroom for the vertical scaling happening at Layer 1, which is exactly what causes eviction risk later in this article.
Layer 4: Multi-tenant governance. ResourceQuotas cap how much a namespace can consume in a multi-tenant Kubernetes environment, which prevents any one team from monopolizing shared capacity. The limits are static, though, and static caps encourage teams to hoard headroom rather than release it, since nobody wants to be the namespace that runs out mid-incident.
| Layer | Standard Tool | What It Does | Where It Falls Short |
|---|---|---|---|
| Pod rightsizing | Vertical Pod Autoscaler (VPA) | Adjusts CPU/memory requests | Requires pod restarts; no cross-cluster context |
| Horizontal scaling | Horizontal Pod Autoscaler (HPA) / KEDA | Scales replica count on metrics | Reactive, it waits for thresholds to be crossed |
| Node provisioning | Karpenter / Cluster Autoscaler | Launches/terminates nodes | Tightly packs nodes, leaving no headroom for vertical scaling |
| Multi-tenancy | ResourceQuotas | Caps namespace resource usage | Static limits encourage hoarding, don’t reflect real usage |
| Pod and node layers | Zesty’s Multi-Dimensional Autoscaling (MDA) and Adaptive Pod Placement (APP) | Rightsizes and autoscales pods, optimizes minReplicas and reduce node fragmentation continuously, in real time | N/A |
None of the four standard layers talk to each other by default. VPA can resize a pod without knowing a node is already packed tight by Karpenter. HPA can add replicas without knowing the namespace is near its ResourceQuota ceiling. Coordinating across layers, rather than tuning each one in isolation, is what determines whether an optimization program holds up under real production traffic. The rollout plan in the next section is built around closing those coordination gaps one phase at a time, instead of trying to fix all four layers on the same day.
A 90-Day Rollout Plan: Audit, Pilot, Automate, Govern
Kubernetes resource optimization works best as a phased rollout rather than a big-bang tool swap. Each phase de-risks the next, so a team never has to make a cluster-wide change without evidence that it holds first. The four phases below map directly onto the four layers described above: audit surfaces where the gaps are, pilot and automate close them at the pod and node level, and govern keeps them closed.
Days 1-15, Audit. Establish a baseline before changing anything. Pull actual CPU and memory usage against allocated requests across every namespace, and flag the services with the widest gap between the two. This phase produces the evidence for every decision that follows: which services are safe to touch first, how much waste is realistically on the table, and which teams are already close to their ResourceQuota ceiling and need extra care during the pilot. Skipping the audit and moving straight to tooling is the single most common reason rollouts stall, since nobody can agree on what “better” looks like without a number to compare against.
Days 16-45, Pilot. Apply pod rightsizing and predictive autoscaling to two or three non-critical services first, ones where a misstep costs a Slack thread instead of an incident. Watch how requests settle over a full weekly cycle, including at least one deploy, before drawing conclusions about savings or stability. This phase also surfaces whether the standard tools from the four-layer model are fighting each other on real traffic, for example whether VPA and HPA are both reacting to the same metric spike, before that conflict reaches a workload that actually matters to customers.
Days 46-75, Automate: Extending Kubernetes Resource Optimization to Production. Extend the pilot’s approach to production workloads, but only once there’s elastic headroom in place to absorb spikes without triggering eviction loops. This is the phase where manual tuning stops scaling. A platform engineer can watch two or three pilot services closely; nobody can watch two hundred production services the same way, which is why this phase is also where teams typically automate rightsizing rather than review dashboards by hand. Node-level headroom matters just as much here as pod-level sizing, since a rightsizing decision that can’t find room on any node just turns into an eviction later.
Days 76-90, Govern. Set usage-based chargeback so teams see the cost of what they actually consume, not what they requested, and pair it with Quality of Service (QoS) policy per team so critical workloads get guaranteed capacity while lower-priority ones absorb the squeeze first. Governance is what keeps the gains from the first three phases from eroding once the initial rollout excitement fades and requests slowly start drifting again as new services launch and old ones get forgotten.
Manually executing all four phases is possible, but it requires continuous engineering attention that most platform teams don’t have to spare once the pilot phase ends. How Zesty automates rightsizing without downtime is built around exactly that gap: it applies the audit-to-govern logic continuously, without a person re-running the analysis every week.
Five Pitfalls That Undo Optimization Gains
A rollout can follow every phase of the plan above and still lose its gains within a few months if these five failure modes go unaddressed. Each one is a way that a well-intentioned optimization step causes a new problem somewhere else in the stack.
Eviction loops from VPA changes with no node headroom. When VPA resizes a pod upward but the node it’s running on has no spare capacity, Kubernetes evicts the pod to reschedule it, and if every node nearby is equally packed, that pod can cycle through eviction and rescheduling repeatedly instead of landing somewhere stable. Teams often trace this back to Karpenter’s own success: the tighter it packs nodes to save cost, the less room VPA has to work with.
Noisy-neighbor pods degrading latency-sensitive services. Densely packed nodes save on infrastructure cost but put unrelated workloads closer together. A batch job spiking CPU on a shared node can steal cycles from a latency-sensitive service sitting next to it, and standard schedulers have no concept of which pods should never share a node unless someone explicitly configures anti-affinity rules for every sensitive workload.
ResourceQuota violations blocking a rightsizing update. When VPA tries to raise a pod’s request and the namespace is already near its quota ceiling, the update fails, and the pod can end up orphaned mid-update, running with neither its old sizing nor its new one cleanly applied. This is a common failure point in multi-tenant Kubernetes environments where quotas were set once and never reexamined against actual usage.
Spot instances used without regard for workload sensitivity. Spot capacity is a legitimate way to cut costs, but applying it indiscriminately, including to workloads that can’t tolerate interruption, turns a savings tactic into an outage risk. Which workloads can absorb a Spot reclaim and which can’t needs to be a deliberate decision, not a blanket policy applied cluster-wide for convenience.
Cost data that lags 24 to 48 hours behind cloud billing. Most cost tooling reports what happened a day or two ago, which makes it useless for catching a misconfigured deployment before it burns through budget over a weekend. By the time a team sees the number, the workload has already been running oversized for two full days, and the fix arrives well after the damage is done.
Zesty’s Adaptive Pod Placement (APP) addresses the node-packing side of this list directly, coordinating placement decisions with real-time usage so pods land on nodes with headroom instead of triggering the eviction and noisy-neighbor problems above.
Metrics That Prove Kubernetes Cost Optimization Is Working
Kubernetes cost optimization needs proof, not a sense that things feel better after a rollout. Without a defined set of metrics, it’s easy to celebrate a good month that was actually just a quiet traffic period, or miss a slow regression until it shows up as a much bigger bill three months later. These are the metrics that separate a program that’s actually working from one that just looks busy:
- Allocated vs. used CPU/memory ratio: target under 20% headroom for non-bursty workloads. Anything consistently higher is still overprovisioning, just at a smaller scale than before the rollout started.
- OOMKilled incident rate: should trend toward zero for workloads with correctly sized memory requests. A rising rate signals limits set too aggressively during rightsizing, and it’s usually the first metric to move when a pilot phase goes wrong.
- Pod eviction count: a leading indicator of node headroom problems before they show up as user-facing incidents. A sudden increase here almost always traces back to node packing getting tighter than the pod-level scaling can tolerate.
- Cost-per-namespace trend: should decline steadily post-rollout, not just in the pilot phase. A trend that flattens after 30 days often means governance policies aren’t holding, or that new namespaces launched without inheriting the same sizing discipline.
- Spot instance interruption rate vs. savings captured: validates whether Spot adoption is actually net-positive for a given workload, or whether interruption costs are steadily eating into the discount.
Tracking these weekly, not quarterly, is what catches drift before it turns into another six months of overprovisioning. A quarterly review only ever explains what already happened; a weekly one catches it while there’s still time to correct course. None of these metrics needs a dashboard built from scratch. They’re a starting checklist for whatever monitoring stack a team already has in place, and the value comes from checking them on a fixed schedule rather than only after someone notices the bill went up.
Conclusion: Automating the Framework
The four-layer model and the 90-day rollout plan both work when run manually. Plenty of platform teams have audited namespace usage by hand, piloted VPA on a couple of services, and set up ResourceQuotas without any outside tooling. The limitation isn’t whether it’s possible. It’s whether a team has the continuous engineering time to keep doing it after the initial rollout, across every namespace, every week, indefinitely, while also keeping up with new services, new release cycles, and the usual list of everything else on a platform team’s plate.
That’s usually where a well-run rollout starts to slip. The audit-to-govern approach still works fine on paper; the people running it simply have a dozen other priorities competing for the same hours it takes to keep rightsizing current across a growing cluster.
That’s the gap Zesty’s Multi-Dimensional Autoscaling (MDA) is built to close at the pod level: continuous rightsizing and autoscaling that adjusts requests and minReplicas in real-time as usage actually shifts, without the manual re-tuning cycle described in the audit and pilot phases above. Paired with Adaptive Pod Placement reducing fragmentation and enabling node consolidation, the two layers stay coordinated instead of working against each other the way VPA, HPA, and Karpenter can when managed separately.
Kubernetes resource optimization stops being a quarterly project and becomes a continuous, cross-layer process once pod-level and node-level decisions are made in real time, together, instead of by separate tools acting on separate schedules. Book a Demo with Zesty to see how that coordination applies to a specific cluster’s usage patterns.
FAQs
What is Kubernetes resource optimization and why does it matter for cloud costs?
Kubernetes resource optimization is the practice of continuously matching pod CPU and memory requests to actual usage, rather than setting them once at deployment. It matters because over-provisioned clusters waste an estimated 60-70% of allocated compute, and that waste compounds every month it goes unaddressed.
