How to Implement StatefulSets with Helm in Kubernetes

How to Implement StatefulSets with Helm in Kubernetes

Deployments are great for stateless applications where any replica is interchangeable, but the moment you need stable network identities, ordered startup, or persistent per-replica storage — databases, message brokers, distributed caches — you need a StatefulSet. Packaging one cleanly and reusably with Helm takes a bit more care than a typical stateless chart. This guide walks through why StatefulSets exist, how they behave differently from Deployments, and how to build a production-ready Helm chart around one.

Why StatefulSets Exist

A Deployment’s Pods are fungible — Pod names are randomly suffixed, storage is typically shared or ephemeral, and Pods can be created/destroyed in any order. That model breaks for anything that needs:

  • Stable, predictable network identity (Pod-0 is always Pod-0, even after a restart).
  • Stable storage per replica — Pod-0 always gets its own PersistentVolumeClaim back, not a random one.
  • Ordered, graceful deployment and scaling — Pod-0 must be Running and Ready before Pod-1 starts, which matters for things like database replication bootstrapping.

A StatefulSet guarantees all three.

Kubernetes Architecture: What Makes a StatefulSet Different

  • Stable identity: Pods are named <statefulset-name>-0, <statefulset-name>-1, etc., not randomly suffixed.
  • Headless Service required: StatefulSets need a Service with clusterIP: None to provide stable DNS entries per Pod (pod-0.service-name.namespace.svc.cluster.local).
  • volumeClaimTemplates: Instead of one shared PVC, each replica gets its own PVC, created from a template, and that PVC follows the same-named Pod across rescheduling.
  • Ordered rolling updates by default: Updates happen in reverse ordinal order (highest number first) unless you configure podManagementPolicy: Parallel.

Step 1: Scaffold a Helm Chart

helm create my-statefulset-app
cd my-statefulset-app
rm templates/deployment.yaml templates/hpa.yaml templates/service.yaml

Step 2: Define the Headless Service

templates/headless-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: {{ include "my-statefulset-app.fullname" . }}-headless
  labels:
    {{- include "my-statefulset-app.labels" . | nindent 4 }}
spec:
  clusterIP: None
  selector:
    {{- include "my-statefulset-app.selectorLabels" . | nindent 4 }}
  ports:
    - name: app
      port: {{ .Values.service.port }}
      targetPort: {{ .Values.service.targetPort }}

Step 3: Define the StatefulSet Template

templates/statefulset.yaml:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: {{ include "my-statefulset-app.fullname" . }}
  labels:
    {{- include "my-statefulset-app.labels" . | nindent 4 }}
spec:
  serviceName: {{ include "my-statefulset-app.fullname" . }}-headless
  replicas: {{ .Values.replicaCount }}
  podManagementPolicy: {{ .Values.podManagementPolicy | default "OrderedReady" }}
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      partition: 0
  selector:
    matchLabels:
      {{- include "my-statefulset-app.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      labels:
        {{- include "my-statefulset-app.selectorLabels" . | nindent 8 }}
    spec:
      terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds | default 30 }}
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          ports:
            - containerPort: {{ .Values.service.targetPort }}
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
          volumeMounts:
            - name: data
              mountPath: {{ .Values.persistence.mountPath }}
          readinessProbe:
            tcpSocket:
              port: {{ .Values.service.targetPort }}
            initialDelaySeconds: 10
            periodSeconds: 5
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: [ {{ .Values.persistence.accessMode | quote }} ]
        storageClassName: {{ .Values.persistence.storageClass }}
        resources:
          requests:
            storage: {{ .Values.persistence.size }}

Step 4: Values File

values.yaml:

replicaCount: 3

image:
  repository: myrepo/statefuldb
  tag: "1.0.0"

service:
  port: 5432
  targetPort: 5432

persistence:
  size: 10Gi
  accessMode: ReadWriteOnce
  storageClass: "fast-ssd"
  mountPath: /var/lib/data

podManagementPolicy: OrderedReady
terminationGracePeriodSeconds: 60

resources:
  requests:
    cpu: 500m
    memory: 1Gi
  limits:
    cpu: "1"
    memory: 2Gi

Step 5: Install and Verify

helm install mydb ./my-statefulset-app -n data --create-namespace
kubectl get statefulset -n data
kubectl get pods -n data -w

Expected Pod creation order:

mydb-0   0/1   Pending
mydb-0   1/1   Running
mydb-1   0/1   Pending
mydb-1   1/1   Running
mydb-2   0/1   Pending
mydb-2   1/1   Running

Each Pod only starts once the previous one is Running and Ready — this is the ordered guarantee at work.

Step 6: Verify Stable Storage and Identity

kubectl get pvc -n data
NAME           STATUS   VOLUME    CAPACITY   ACCESS MODES
data-mydb-0    Bound    pvc-abc   10Gi       RWO
data-mydb-1    Bound    pvc-def   10Gi       RWO
data-mydb-2    Bound    pvc-ghi   10Gi       RWO

Delete mydb-1 and watch it come back with the same PVC, not a fresh empty volume:

kubectl delete pod mydb-1 -n data
kubectl get pod mydb-1 -n data -o jsonpath='{.spec.volumes[0].persistentVolumeClaim.claimName}'
# data-mydb-1

Check DNS resolution from another Pod in the cluster:

kubectl run -it --rm debug --image=busybox -n data -- nslookup mydb-1.mydb-headless.data.svc.cluster.local

Rolling Updates with Partitions

For careful, staged rollouts (e.g., testing a new version on just the highest-ordinal replica first), use rollingUpdate.partition:

updateStrategy:
  type: RollingUpdate
  rollingUpdate:
    partition: 2

With partition: 2 on a 3-replica set, only mydb-2 gets updated on a chart upgrade; mydb-0 and mydb-1 stay untouched until you lower the partition value. This is a standard canary pattern for stateful workloads where you want to validate before wider rollout.

helm upgrade mydb ./my-statefulset-app --set image.tag=1.1.0 --set statefulset.partition=2
# validate mydb-2, then:
helm upgrade mydb ./my-statefulset-app --set image.tag=1.1.0 --set statefulset.partition=0

Scaling Considerations

Scaling up adds new ordinals sequentially (mydb-3 after mydb-0..2 are healthy); scaling down removes the highest ordinal first, not an arbitrary one. This matters for anything using replica index for identity, like a database where node 0 is always primary by convention.

kubectl scale statefulset mydb --replicas=5 -n data

Note that scaling down does not delete the PVCs by default — they’re retained so you don’t lose data by accident. Clean them up explicitly if you actually want the storage gone:

kubectl delete pvc data-mydb-4 data-mydb-3 -n data

Security and Best Practices

  • Set podManagementPolicy: Parallel only for stateless-within-a-StatefulSet cases where ordering genuinely doesn’t matter (e.g., you just want stable identity/storage without startup ordering) — most real stateful apps need OrderedReady, the default.
  • Always define a readiness probe that reflects actual application health (e.g., can accept queries, has joined a cluster), not just “process is running” — this is what the ordering guarantee depends on.
  • Use PodDisruptionBudgets alongside StatefulSets to protect quorum-based systems (etcd, Kafka, Cassandra) from losing too many replicas during node maintenance.
  • Set terminationGracePeriodSeconds high enough for graceful shutdown (e.g., a database flushing to disk) — the default 30s is often too short for real workloads.
  • For disaster recovery, remember volumeClaimTemplates only protect against Pod rescheduling, not storage backend failure — pair with regular backups appropriate to your storage class/provider.

Common Mistakes

  • Forgetting the headless Service — without it, per-Pod DNS simply won’t resolve, breaking peer discovery for clustered apps.
  • Assuming scaling down deletes PVCs automatically — it doesn’t, which is a safety feature but surprises people expecting Deployment-like cleanup behavior.
  • Using emptyDir instead of volumeClaimTemplates for state that needs to survive Pod rescheduling.
  • Setting overly aggressive readiness probes that flap, causing the ordered rollout to stall waiting for a Pod that’s actually healthy but misreporting.

Summary

StatefulSets solve a real problem that Deployments can’t: stable identity, stable storage, and ordered lifecycle management for genuinely stateful workloads. Packaging one in Helm mainly means being deliberate about the headless Service, volumeClaimTemplates, and update/scaling behavior — none of which are optional details, since getting them wrong breaks the exact guarantees you chose a StatefulSet for in the first place.

References

Total
7
Shares

Leave a Reply

Previous Post
How to Set Up Kubernetes Monitoring with Zabbix

How to Set Up Kubernetes Monitoring with Zabbix

Next Post
How to Set Up Pod Disruption Budgets with Thanos in Kubernetes

How to Set Up Pod Disruption Budgets with Thanos in Kubernetes

Related Posts