Manually adjusting replica counts based on traffic patterns is one of those tasks that feels manageable until it isn’t — a marketing campaign goes viral at 11 p.m. and suddenly you’re scrambling to scale up before the site falls over. Horizontal Pod Autoscaling (HPA) automates exactly this, watching metrics like CPU or memory usage and adjusting replica counts in real time. In this guide, I’ll cover how HPA works under the hood, set up a working example with the metrics server, and go beyond CPU into custom metrics.
How HPA Works
The HPA controller runs a control loop (by default every 15 seconds) that:
- Queries current metrics for the target Pods (CPU, memory, or custom/external metrics).
- Compares them against the target value you defined.
- Calculates the desired replica count using the formula:
desiredReplicas = ceil(currentReplicas * (currentMetricValue / desiredMetricValue)). - Updates the Deployment’s replica count accordingly, respecting min/max bounds.
Importantly, HPA scales Deployments, ReplicaSets, or StatefulSets — not individual Pods directly — and it requires actual metrics data to function, which means you need the Metrics Server (or a custom metrics adapter) installed in your cluster.
Step 1: Install the Metrics Server
Most managed clusters (EKS, GKE, AKS) have this available as an add-on, but if not:
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
Verify it’s collecting data:
kubectl top nodes
kubectl top pods -n production
Expected output:
NAME CPU(cores) MEMORY(bytes)
myapp-6d9f8b7c5d-4k2pl 120m 256Mi
myapp-6d9f8b7c5d-9xqwt 95m 240Mi
If kubectl top returns nothing, HPA won’t have data to work with, so this step is non-negotiable.
Step 2: Set Resource Requests on Your Deployment
HPA calculates CPU/memory percentages relative to the requests defined on your containers, not limits. Without requests set, HPA has nothing to compare against:
# myapp-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
replicas: 2
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: myrepo/myapp:1.0.0
resources:
requests:
cpu: 200m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi
kubectl apply -f myapp-deployment.yaml
Step 3: Create the HorizontalPodAutoscaler
# myapp-hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
kubectl apply -f myapp-hpa.yaml
kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
myapp-hpa Deployment/myapp 35%/60% 2 10 2 30s
This means: keep average CPU utilization across all Pods around 60% of their requested 200m, scaling between 2 and 10 replicas as needed.
Alternatively, create the same thing imperatively for quick testing:
kubectl autoscale deployment myapp --cpu-percent=60 --min=2 --max=10
Step 4: Watch It Scale Under Load
Generate load to see it in action:
kubectl run load-generator --rm -it --image=busybox -- /bin/sh -c \
"while true; do wget -q -O- http://myapp-service; done"
In another terminal, watch the HPA react:
kubectl get hpa myapp-hpa --watch
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
myapp-hpa Deployment/myapp 142%/60% 2 10 2 2m
myapp-hpa Deployment/myapp 142%/60% 2 10 5 2m30s
myapp-hpa Deployment/myapp 58%/60% 2 10 5 3m30s
Once load stops, the HPA scales back down, though more conservatively — by default there’s a five-minute stabilization window on scale-down to avoid flapping.
Step 5: Scaling on Memory or Multiple Metrics
You can target memory instead of, or alongside, CPU:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 75
When multiple metrics are defined, HPA computes the desired replica count for each and picks the largest value — meaning it scales to satisfy whichever metric needs the most capacity.
Step 6: Scaling on Custom Metrics (e.g., Requests Per Second)
CPU isn’t always the right signal — for a queue worker, you probably care about queue length; for an API, maybe requests per second. This requires a metrics adapter like Prometheus Adapter, exposing custom metrics through the custom.metrics.k8s.io API:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa-custom
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 2
maxReplicas: 20
metrics:
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: "50"
This requires Prometheus scraping your app’s metrics endpoint and the Prometheus Adapter configured to expose http_requests_per_second as a Kubernetes custom metric — a more advanced setup, but far more accurate for many real workloads than CPU alone.
Step 7: Tuning Scaling Behavior
Newer HPA versions let you fine-tune scale-up/scale-down speed and stabilization windows to avoid thrashing:
spec:
behavior:
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 50
periodSeconds: 60
scaleUp:
stabilizationWindowSeconds: 0
policies:
- type: Percent
value: 100
periodSeconds: 15
- type: Pods
value: 4
periodSeconds: 15
selectPolicy: Max
This configuration scales up aggressively (doubling Pods or adding 4, whichever is more, every 15 seconds) but scales down conservatively (max 50% reduction per minute, waiting 5 minutes of stability first) — a pattern I use for anything customer-facing where sudden capacity loss is riskier than temporarily over-provisioning.
Debugging HPA Issues
kubectl describe hpa myapp-hpa
Common issues surfaced here:
Warning FailedGetResourceMetric 30s horizontal-pod-autoscaler
failed to get cpu utilization: unable to get metrics for resource cpu:
no metrics returned from resource metrics API
This almost always means the Metrics Server isn’t running or reachable, or the target Pods don’t have resource requests defined.
Best Practices
- Always set
minReplicasabove 1 for production workloads to maintain availability during rolling updates. - Combine HPA with a PodDisruptionBudget so scale-down events and node maintenance don’t drop below your minimum availability.
- Combine with Cluster Autoscaler (or Karpenter on AWS) so there’s actually node capacity available for HPA to schedule new Pods onto — HPA alone doesn’t provision nodes.
- Base target utilization on real load-testing data, not arbitrary defaults like 50% or 80%.
- For latency-sensitive services, consider custom metrics like request latency or queue depth rather than relying solely on CPU, which often lags behind real user impact.
Common Mistakes
- Missing resource requests, leaving HPA with nothing to calculate against.
- Setting
maxReplicastoo low, silently capping scaling during real traffic spikes. - Ignoring the interaction between HPA and Cluster Autoscaler, leading to Pods stuck
Pendingbecause there’s no node capacity. - Testing HPA behavior in a low-traffic dev cluster and assuming production tuning will be identical.
HPA and Cluster Autoscaler Working Together
It’s worth being explicit about a relationship that confuses a lot of people early on: HPA and Cluster Autoscaler solve two different problems that happen to compound. HPA decides how many Pods your application needs based on observed metrics. Cluster Autoscaler (or Karpenter) decides how many nodes the cluster needs, based on whether existing Pods (including new ones HPA just requested) can actually be scheduled. If HPA scales a Deployment from 3 to 12 replicas but your nodes only have room for 8, the remaining 4 stay Pending until Cluster Autoscaler notices and provisions more capacity — which itself takes anywhere from 30 seconds to a few minutes depending on cloud provider and instance type availability. For traffic spikes that arrive faster than node provisioning can keep up, consider running some baseline overprovisioned capacity (a low-priority “pause” Deployment that Cluster Autoscaler can evict to make room instantly) so there’s always a buffer of ready nodes rather than starting from zero spare capacity every time.
apiVersion: apps/v1
kind: Deployment
metadata:
name: overprovisioning
spec:
replicas: 3
selector:
matchLabels:
app: overprovisioning
template:
metadata:
labels:
app: overprovisioning
spec:
priorityClassName: overprovisioning
containers:
- name: pause
image: registry.k8s.io/pause:3.9
resources:
requests:
cpu: 500m
memory: 512Mi
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: overprovisioning
value: -1
globalDefault: false
These low-priority “placeholder” Pods reserve node capacity but get preempted instantly whenever real workloads need the room, which effectively gives you a standing buffer of already-provisioned nodes without paying for genuinely idle infrastructure indefinitely.
HPA Against External Metrics
Beyond Pod-level custom metrics, HPA also supports External metric types, useful when the signal that should drive scaling doesn’t come from inside the cluster at all — a cloud provider’s queue depth (SQS, Pub/Sub) is the classic example for scaling worker Deployments:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: worker-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: queue-worker
minReplicas: 1
maxReplicas: 30
metrics:
- type: External
external:
metric:
name: sqs_queue_depth
selector:
matchLabels:
queue: myapp-jobs
target:
type: AverageValue
averageValue: "100"
This requires an external metrics adapter (like KEDA, which has become the popular choice for exactly this kind of event-driven scaling) exposing the metric through the external.metrics.k8s.io API. KEDA in particular is worth a look if most of your scaling triggers are queue- or event-based rather than CPU-based — it wraps HPA with a much larger library of ready-made “scalers” for common systems (SQS, Kafka, RabbitMQ, Prometheus queries, and dozens more) so you don’t have to build a custom metrics adapter yourself.
Scale-to-Zero
Standard HPA can’t scale below 1 replica — minReplicas must be at least 1. For workloads that should genuinely scale to zero when idle (an infrequently-used internal tool, an event-driven worker with long idle periods), KEDA again fills this gap, handling the 0-to-1 transition itself by watching the trigger source directly and creating the first replica on demand, then handing off to standard HPA-style scaling once traffic is flowing.
Summary
Horizontal Pod Autoscaling automates one of the most operationally tedious parts of running Kubernetes: matching capacity to demand in real time. Getting it right requires the Metrics Server, properly set resource requests, a sensible target utilization, and — for production-grade setups — tuned scaling behavior and possibly custom metrics beyond plain CPU. Once configured, it quietly handles traffic spikes that would otherwise mean a 2 a.m. page.
