Helm makes deploying applications easy, and that ease is exactly what makes secrets management with Helm dangerous if you’re not careful. I’ve seen (and, early in my career, personally caused) database passwords sitting in plaintext inside a values.yaml file that got committed to a public repo. Helm doesn’t encrypt anything by default — it just templates YAML — so handling secrets safely requires deliberately layering additional tooling on top. This article walks through the native Kubernetes Secret object, how Helm templates them, and the production-grade patterns that avoid plaintext secrets in your Git history.
How Kubernetes Secrets Work Internally
A Kubernetes Secret is stored in etcd, base64-encoded (not encrypted, by default) unless you’ve configured encryption at rest on the API server. Secrets are exposed to pods either as environment variables or mounted files via a secretRef or volumeMount.
apiVersion: v1
kind: Secret
metadata:
name: db-credentials
namespace: production
type: Opaque
data:
username: YWRtaW4=
password: c3VwZXJzZWNyZXQ=
Base64 is encoding, not encryption — anyone with get secret RBAC access can trivially decode it. This matters a lot for how you think about RBAC scoping and etcd encryption, covered later.
The Naive (and Risky) Helm Approach
The simplest way to template a Secret in a Helm chart looks like this:
# templates/secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: {{ .Release.Name }}-db-credentials
type: Opaque
stringData:
username: {{ .Values.db.username }}
password: {{ .Values.db.password }}
# values.yaml
db:
username: admin
password: supersecret # DO NOT DO THIS
The problem: values.yaml ends up in Git, in Helm release history (helm get values), and potentially in CI logs. This is the exact anti-pattern that causes credential leaks.
Pattern 1: --set for Ephemeral Secrets (CI/CD Injection)
A slightly better pattern avoids committing the value at all, injecting it only at deploy time from a CI secret store:
helm upgrade --install myapp ./mychart \
--set db.password="$DB_PASSWORD" \
--namespace production
This keeps the plaintext out of Git, but it’s still visible in helm history and process lists unless you’re careful, and CI environment variables need to be sourced from a proper secrets manager (GitHub Actions secrets, GitLab CI variables, Vault, etc.) rather than hardcoded in pipeline YAML.
Pattern 2: External Secrets Operator (Recommended)
The pattern I actually use in production decouples secret values from Helm entirely. Helm only creates a reference (an ExternalSecret object); the actual secret material lives in a dedicated secrets manager (AWS Secrets Manager, HashiCorp Vault, GCP Secret Manager) and is synced into the cluster by the External Secrets Operator.
Install the operator:
helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets external-secrets/external-secrets \
--namespace external-secrets \
--create-namespace
Define a SecretStore pointing at your backend (AWS Secrets Manager example):
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: aws-secrets-manager
namespace: production
spec:
provider:
aws:
service: SecretsManager
region: us-east-1
auth:
jwt:
serviceAccountRef:
name: external-secrets-sa
Now, instead of a raw Secret template in your Helm chart, template an ExternalSecret:
# templates/external-secret.yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: {{ .Release.Name }}-db-credentials
namespace: {{ .Release.Namespace }}
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secrets-manager
kind: SecretStore
target:
name: {{ .Release.Name }}-db-credentials
creationPolicy: Owner
data:
- secretKey: username
remoteRef:
key: production/db-credentials
property: username
- secretKey: password
remoteRef:
key: production/db-credentials
property: password
helm upgrade --install myapp ./mychart --namespace production
kubectl get externalsecret -n production
kubectl get secret myapp-db-credentials -n production
Now values.yaml never contains a single secret value — the actual credential lives only in AWS Secrets Manager, and the operator handles syncing and rotation on refreshInterval.
Pattern 3: Sealed Secrets (GitOps-Friendly Alternative)
If you want fully GitOps-committable secrets without an external secrets manager dependency, Bitnami’s Sealed Secrets encrypts secret values client-side so only the in-cluster controller can decrypt them.
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm install sealed-secrets sealed-secrets/sealed-secrets --namespace kube-system
Encrypt a Secret locally with kubeseal:
kubectl create secret generic db-credentials \
--from-literal=username=admin \
--from-literal=password=supersecret \
--dry-run=client -o yaml | kubeseal --format yaml > sealed-secret.yaml
The resulting SealedSecret is safe to commit and template through Helm, since it’s only decryptable by the controller’s private key inside the cluster:
# templates/sealed-secret.yaml
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
name: {{ .Release.Name }}-db-credentials
namespace: {{ .Release.Namespace }}
spec:
encryptedData:
username: AgBy3i4OJSWK+PiTySYZZA9rO43cGDEQ...
password: AgCtL9r4h2ZVKmL8pQ3XyZ...
helm upgrade --install myapp ./mychart --namespace production
kubectl get sealedsecret -n production
kubectl get secret myapp-db-credentials -n production # created automatically by the controller
Pattern 4: Helm Secrets Plugin with SOPS
For teams that want encrypted values files directly in the Helm workflow (rather than separate CRDs), the helm-secrets plugin combined with Mozilla SOPS encrypts values.yaml fields in place:
helm plugin install https://github.com/jkroepke/helm-secrets
sops --encrypt --kms arn:aws:kms:us-east-1:123456789:key/abc secrets.yaml > secrets.enc.yaml
helm secrets upgrade --install myapp ./mychart -f secrets.enc.yaml
This keeps the encrypted file committable to Git while decryption happens transparently at deploy time using KMS-backed keys.
RBAC and Etcd Encryption
Regardless of which pattern you use, two cluster-level protections matter:
- Enable encryption at rest for Secrets in the API server so etcd doesn’t store plaintext base64 blobs unencrypted on disk:
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
- secrets
providers:
- aescbc:
keys:
- name: key1
secret: <base64-encoded-32-byte-key>
- identity: {}
- Scope RBAC tightly —
get/listonsecretsshould never be granted broadly:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: secret-reader
namespace: production
rules:
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["myapp-db-credentials"]
verbs: ["get"]
Rotating Secrets Without Downtime
Rotation is where a lot of Helm-based secret setups fall apart in practice, since simply changing a value and running helm upgrade can cause a brief window of inconsistency if the application doesn’t reload the new secret gracefully. A few patterns that work well:
Checksum annotation to force pod restarts on secret change:
# templates/deployment.yaml
spec:
template:
metadata:
annotations:
checksum/secret: {{ include (print $.Template.BasePath "/external-secret.yaml") . | sha256sum }}
This forces a rolling restart whenever the rendered Secret content changes, ensuring pods always pick up the latest credential rather than caching a stale one in memory indefinitely.
External Secrets Operator’s built-in refresh: with refreshInterval: 1h set on the ExternalSecret object (as shown earlier), the operator polls the backend and updates the in-cluster Secret automatically — but note this alone doesn’t restart pods; you still need the checksum annotation pattern above, or application-level hot-reload logic, to actually pick up the new value.
Coordinated rotation for database credentials specifically usually needs a brief dual-credential window: create the new credential in the database, update the secret, roll pods, confirm all pods are using the new credential, then revoke the old one. Doing this atomically without a short overlap window risks an outage if any pod is mid-restart when the old credential is revoked.
Verifying No Secrets Leaked Into Git History
Even with the patterns above, it’s worth periodically scanning your repository for accidentally committed secrets, since a single early mistake can persist in Git history long after the immediate values.yaml was cleaned up:
pip install detect-secrets --break-system-packages
detect-secrets scan --all-files > .secrets.baseline
Or with gitleaks, which is purpose-built for this and works well as a pre-commit hook or CI gate:
gitleaks detect --source . --verbose
# .github/workflows/security.yml
- name: Scan for leaked secrets
uses: gitleaks/gitleaks-action@v2
If something is found in history, rotating the credential is the only real fix — rewriting Git history to remove it doesn’t help once it’s been pushed to a shared remote, since it may already be cached or cloned elsewhere.
Common Mistakes
- Committing
values.yamlwith real secret values — the single most common cause of leaked credentials I’ve seen in Helm-based deployments. - Assuming base64 encoding is encryption and treating Secrets as safe to expose via
kubectl describeor logging pipelines. - Not rotating secrets after a suspected leak — Helm doesn’t automatically rotate anything; that’s on you or your secrets manager’s rotation policy.
- Mixing Helm’s Secret templating with manually
kubectl apply‘d secrets, causing drift wherehelm upgradeoverwrites manually rotated credentials.
Choosing Between the Patterns
With four approaches covered, it’s worth a quick decision guide based on what I’ve actually seen work for different team sizes and constraints:
- External Secrets Operator — best default choice if you already use a cloud secrets manager (AWS Secrets Manager, GCP Secret Manager, Vault); centralizes rotation and audit logging where it belongs, outside Kubernetes entirely.
- Sealed Secrets — best for GitOps-heavy teams that want everything, including secrets, committable to Git without standing up a separate secrets manager service.
- Helm Secrets + SOPS — a good middle ground when you want encrypted values files that fit naturally into an existing Helm-centric workflow without adopting new CRDs.
- Plain
--setwith CI-injected values — acceptable for smaller teams or less sensitive environments, but I’d treat it as a starting point to graduate away from rather than a long-term production pattern.
Summary
Helm itself has no secrets management opinion — it’s just a templating engine, so treating values.yaml as a safe place for credentials is the core mistake to avoid. The production-grade path is decoupling secret material from Helm charts entirely: External Secrets Operator for teams with a cloud secrets manager, Sealed Secrets or SOPS for GitOps-committable encrypted values. Pair either with etcd encryption at rest and tight RBAC, and you’ve closed the most common leakage paths.