How to Implement Pod Priority and Preemption with Helm in Kubernetes

When a cluster is under resource pressure, not all workloads are equal. A batch job that can retry later matters a lot less than the payment-processing service that needs to stay up. Kubernetes handles this trade-off through Pod Priority and Preemption — and when you’re deploying real applications with Helm, you want this configured consistently as part of your chart, not bolted on manually after the fact. In this guide, I’ll cover how priority and preemption work, and how to wire them into a Helm-based deployment.

What Priority and Preemption Actually Do

Every Pod can carry a priority — an integer value derived from a PriorityClass object. When the scheduler can’t find a node with enough resources for a pending high-priority Pod, it may preempt (evict) lower-priority Pods on some node to make room, rather than leaving the high-priority Pod stuck in Pending indefinitely.

This is separate from and complementary to Pod Disruption Budgets — the scheduler respects PDBs where possible during preemption, but a PDB does not fully protect a Pod from being preempted if there’s no other way to schedule a higher-priority Pod.

Kubernetes Architecture: Where This Happens

Priority and preemption live entirely in the scheduler:

1. A Pod is submitted with a priorityClassName.
2. The admission controller resolves that name to a numeric priority value, stamped onto the Pod spec.
3. The scheduler tries to place it normally first.
4. If no node fits, and preemption is enabled (default), the scheduler looks for a node where evicting some lower-priority Pods would make room, chooses the node with the least “collateral damage,” and evicts just enough Pods.
5. The preempted Pods go back to Pending and get rescheduled elsewhere if capacity exists.

Step 1: Define PriorityClasses

PriorityClasses are cluster-scoped, so they’re usually created once, separately from your application charts (or as a shared “platform” chart installed first).

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: critical-priority
value: 1000000
globalDefault: false
description: "Reserved for critical production services."
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: standard-priority
value: 100000
globalDefault: true
description: "Default priority for most application workloads."
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: batch-priority
value: 1000
globalDefault: false
preemptionPolicy: Never
description: "Low priority for batch and background jobs. Never preempts others."

kubectl apply -f priorityclasses.yaml
kubectl get priorityclass

Notice preemptionPolicy: Never on the batch class — this means Pods in that class will never trigger preemption of others, even though they still have a defined (low) priority. That’s a useful pattern for jobs that should wait patiently rather than kick anything else off a node.

Step 2: Reference PriorityClass in a Helm Chart

Rather than hardcoding priorityClassName in every Deployment template, expose it as a configurable value so different environments (or different releases of the same chart) can set it appropriately.

values.yaml:

priorityClassName: standard-priority

replicaCount: 3

resources:
  requests:
    cpu: 250m
    memory: 256Mi
  limits:
    cpu: 500m
    memory: 512Mi

templates/deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}-app
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ .Release.Name }}
  template:
    metadata:
      labels:
        app: {{ .Release.Name }}
    spec:
      {{- if .Values.priorityClassName }}
      priorityClassName: {{ .Values.priorityClassName }}
      {{- end }}
      containers:
        - name: app
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          resources:
            {{- toYaml .Values.resources | nindent 12 }}

Step 3: Install with an Override per Environment

For a critical production release:

helm install payments-api ./chart -n production \
  --set priorityClassName=critical-priority

For a batch workload chart:

helm install nightly-report ./chart -n batch \
  --set priorityClassName=batch-priority

This gives you one chart, reused across workloads with wildly different scheduling importance, without duplicating templates.

Step 4: Verify Priority Assignment

kubectl get pod payments-api-xyz -o jsonpath='{.spec.priorityClassName}{"\n"}{.spec.priority}'

Expected output:

critical-priority
1000000

Observing Preemption in Action

When preemption occurs, check events on the newly scheduled Pod:

kubectl describe pod payments-api-xyz

You’ll see an event like:

Events:
  Type     Reason      Message
  ----     ------      -------
  Normal   Preempted   Preempted by payments-api-xyz on node worker-3

And on the evicted Pod’s side, it goes back to Pending, with a Preempting condition explaining why. The kube-scheduler logs (if you have access) also record the preemption decision in detail — useful for post-incident review of why a batch job got killed mid-run.

Handling Graceful Termination During Preemption

Preempted Pods still respect terminationGracePeriodSeconds, so a preempted Pod isn’t just SIGKILLed instantly — it gets its normal shutdown hooks. For workloads doing meaningful in-flight work, tune this appropriately in your chart:

spec:
  template:
    spec:
      terminationGracePeriodSeconds: 60

Best Practices

• Don’t overuse high priority. If every team sets their workloads to the highest PriorityClass “just in case,” you lose the entire mechanism’s value — it becomes a race to the top rather than a meaningful signal.
• Reserve the top priority tier (e.g., system-cluster-critical, which Kubernetes itself uses for core components like kube-dns) for things that are genuinely cluster-critical; don’t let application teams use system-reserved classes.
• Combine with resource requests/limits and PodDisruptionBudgets — priority determines who gets preempted; PDBs limit how much can be preempted from a given set at once, giving you defense in depth.
• Set preemptionPolicy: Never on batch/best-effort classes so they never disrupt others, even though they can still be scheduled when capacity allows.
• Document your PriorityClass tiers in your platform’s onboarding docs so teams pick the correct one instead of guessing.

Common Mistakes

• Forgetting globalDefault: true must be set on exactly one PriorityClass — if none is marked default, unlabeled Pods get priority 0, which is lower than almost everything and can cause them to be preempted unexpectedly.
• Setting overly aggressive high-priority classes on non-critical workloads, causing cascading preemptions across the cluster during load spikes.
• Not testing preemption behavior in a staging cluster before rolling PriorityClasses into production — the first real preemption event shouldn’t be a surprise.

Summary

Pod Priority and Preemption give Kubernetes a principled way to decide who keeps running when resources get tight, and wiring priorityClassName into your Helm charts as a configurable value lets you apply consistent, environment-aware scheduling policy without duplicating manifests. Define a small, well-documented set of PriorityClasses, expose the choice through Helm values, and pair it with resource requests and PDBs for a cluster that degrades gracefully under pressure instead of falling over.

References

• Kubernetes Documentation: Pod Priority and Preemption
• Helm Documentation
• Kubernetes Scheduling Concepts
• CNCF Kubernetes Project