At some point, almost every organization running Kubernetes faces the same question: do we give every team their own cluster, or do we share one cluster across teams and figure out isolation? Both are legitimate answers, but “one cluster per team” gets expensive and operationally heavy fast, so most organizations end up building some form of multi-tenancy into a shared cluster. In this guide, I’ll walk through the real architectural choices, the isolation mechanisms available, and a concrete production setup on AWS EKS.
What “Multi-Tenancy” Actually Means in Kubernetes
Multi-tenancy isn’t one feature — it’s a combination of several isolation layers, and how far you go depends on your trust model:
- Soft multi-tenancy — tenants are trusted (internal teams within the same company), isolation is mostly about preventing accidents and resource contention, not malicious behavior.
- Hard multi-tenancy — tenants are untrusted (e.g., a SaaS platform running customer workloads), and isolation needs to hold up against deliberate attempts to break out or interfere with other tenants.
Kubernetes namespaces alone provide soft multi-tenancy at best. For hard multi-tenancy, you generally need additional layers — separate clusters, or node-level isolation like AWS Fargate or gVisor sandboxing.
Tenancy Models
Model 1: Namespace-per-Tenant (Shared Cluster)
The most common and cost-efficient model for soft multi-tenancy. Each tenant gets a namespace (or set of namespaces), isolated via RBAC, ResourceQuotas, and NetworkPolicies.
Model 2: Cluster-per-Tenant
Each tenant gets a fully dedicated EKS cluster. Maximum isolation, maximum operational overhead — you’re now managing N clusters instead of one. Common for regulated industries or when tenants have genuinely incompatible compliance requirements.
Model 3: Virtual Clusters (vCluster)
A middle ground — lightweight virtual Kubernetes API servers running inside namespaces of a shared host cluster, giving each tenant something that looks and feels like their own cluster (their own API server, their own CRDs, their own RBAC root) without the cost of a fully separate physical cluster.
I’ll focus mainly on Model 1 since it’s the most widely applicable, with notes on where Models 2 and 3 make more sense.
Building Namespace-per-Tenant Isolation
Step 1: Namespace with Labels and Pod Security Standards
apiVersion: v1
kind: Namespace
metadata:
name: tenant-acme
labels:
tenant: acme
tier: standard
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/enforce-version: latest
Step 2: RBAC Scoped to the Tenant
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: tenant-acme
name: tenant-admin
rules:
- apiGroups: ["", "apps", "batch", "networking.k8s.io"]
resources: ["*"]
verbs: ["*"]
- apiGroups: [""]
resources: ["resourcequotas", "limitranges"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: tenant-acme-admins
namespace: tenant-acme
subjects:
- kind: Group
name: acme-admins
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: tenant-admin
apiGroup: rbac.authorization.k8s.io
Note this deliberately excludes resourcequotas and limitranges from the tenant’s write access — tenants shouldn’t be able to raise their own resource ceilings.
On EKS, map IAM identities to these RBAC groups using EKS access entries:
aws eks create-access-entry \
--cluster-name production-cluster \
--principal-arn arn:aws:iam::123456789012:role/acme-team-role \
--kubernetes-groups acme-admins
Step 3: ResourceQuota and LimitRange per Tenant
apiVersion: v1
kind: ResourceQuota
metadata:
name: tenant-acme-quota
namespace: tenant-acme
spec:
hard:
requests.cpu: "10"
requests.memory: 20Gi
limits.cpu: "20"
limits.memory: 40Gi
pods: "30"
persistentvolumeclaims: "5"
services.loadbalancers: "1"
count/ingresses.networking.k8s.io: "5"
apiVersion: v1
kind: LimitRange
metadata:
name: tenant-acme-limits
namespace: tenant-acme
spec:
limits:
- type: Container
default:
cpu: 250m
memory: 256Mi
defaultRequest:
cpu: 100m
memory: 128Mi
max:
cpu: "2"
memory: 4Gi
Step 4: Network Isolation Between Tenants
Default-deny cross-namespace traffic, then explicitly allow only what’s needed:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: deny-cross-tenant
namespace: tenant-acme
spec:
podSelector: {}
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
tenant: acme
- namespaceSelector:
matchLabels:
tier: shared-services
This allows traffic only from pods within the same tenant namespace or from a designated shared-services namespace (for things like a shared ingress controller or shared internal APIs), blocking everything else by default. On EKS, this requires a CNI that enforces NetworkPolicy — Calico, Cilium, or the AWS VPC CNI’s native NetworkPolicy support in recent versions.
Step 5: Enforce Guardrails with Kyverno
RBAC and quotas handle “what can tenants do to their own resources,” but you also need to prevent tenants from doing things that are technically permitted by RBAC but violate platform conventions — like forgetting resource limits, or using a :latest image tag in production.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-tenant-label
spec:
validationFailureAction: Enforce
rules:
- name: check-tenant-label
match:
any:
- resources:
kinds:
- Deployment
- StatefulSet
namespaces:
- "tenant-*"
validate:
message: "All workloads must carry a 'tenant' label matching their namespace."
pattern:
metadata:
labels:
tenant: "?*"
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: disallow-latest-tag
spec:
validationFailureAction: Enforce
rules:
- name: require-image-tag
match:
any:
- resources:
kinds:
- Pod
validate:
message: "Using ':latest' image tag is not allowed."
pattern:
spec:
containers:
- image: "!*:latest"
Step 6: Isolating Compute (Node-Level Separation)
For soft multi-tenancy, sharing nodes across tenants is usually fine. For tenants with stricter isolation needs (compliance, noisy-neighbor sensitivity, or genuinely semi-trusted workloads), dedicate node groups:
apiVersion: v1
kind: Node
metadata:
labels:
tenant: acme
spec:
taints:
- key: tenant
value: acme
effect: NoSchedule
spec:
template:
spec:
tolerations:
- key: tenant
operator: Equal
value: acme
effect: NoSchedule
nodeSelector:
tenant: acme
On EKS, this maps naturally to a dedicated managed node group with a matching taint, or a Karpenter NodePool scoped to a tenant:
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
name: tenant-acme
spec:
template:
metadata:
labels:
tenant: acme
spec:
taints:
- key: tenant
value: acme
effect: NoSchedule
requirements:
- key: kubernetes.io/arch
operator: In
values: ["amd64"]
limits:
cpu: "100"
memory: 200Gi
For genuinely untrusted workloads, go further with AWS Fargate profiles (kernel-level isolation via Firecracker microVMs per pod) instead of shared EC2 nodes:
# Fargate profile (created via eksctl or Terraform, not a k8s manifest)
eksctl create fargateprofile \
--cluster production-cluster \
--name tenant-acme-fargate \
--namespace tenant-acme
Cost Allocation Across Tenants
A practical multi-tenancy concern that’s easy to overlook: knowing what each tenant actually costs. Consistent labeling plus a cost-visibility tool closes this loop:
helm repo add kubecost https://kubecost.github.io/cost-analyzer/
helm install kubecost kubecost/cost-analyzer --namespace kubecost --create-namespace
With consistent tenant labels applied via the Kyverno policy above, Kubecost (or AWS Cost and Usage Reports combined with Kubernetes cost allocation tags) can break down spend per tenant automatically.
Virtual Clusters for Stronger Self-Service Isolation
If tenants need their own CRDs, their own cluster-scoped RBAC root, or genuinely can’t share a single Kubernetes API surface without stepping on each other, vcluster is worth evaluating:
helm repo add loft https://charts.loft.sh
helm install acme-vcluster loft/vcluster --namespace tenant-acme-vcluster --create-namespace
This creates a nested, lightweight Kubernetes control plane inside the host namespace — tenants get kubectl access that behaves like a real dedicated cluster, while still scheduling pods onto the shared underlying node pool.
Admission Control as the Tenancy Enforcement Layer
Everything discussed so far — RBAC, quotas, network policies — is enforced at different layers, which means a genuinely robust multi-tenant platform needs a single admission control layer tying them together and catching anything the other layers miss. This is where Kyverno or OPA Gatekeeper earn their keep as the platform team’s actual enforcement backbone, rather than an optional add-on:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: enforce-tenant-namespace-scoping
spec:
validationFailureAction: Enforce
rules:
- name: block-cross-tenant-service-account-refs
match:
any:
- resources:
kinds:
- RoleBinding
namespaces:
- "tenant-*"
validate:
message: "RoleBindings in tenant namespaces may only reference ServiceAccounts in the same namespace."
deny:
conditions:
all:
- key: "{{ request.object.subjects[].namespace }}"
operator: AnyNotIn
value: ["{{ request.namespace }}"]
This kind of rule catches a subtle but real risk: a tenant creating a RoleBinding that grants access to a ServiceAccount living in a different tenant’s namespace — technically valid RBAC syntax, but a clear tenancy violation that plain RBAC scoping alone won’t prevent, since RBAC doesn’t inherently understand “tenant” as a concept.
Self-Service Tenant Onboarding
At scale, manually creating namespace bundles for every new team doesn’t hold up. A common pattern is a Tenant custom resource that a platform team’s controller (built with something like Kubebuilder, or using a generic tool like Crossplane) reconciles into the full bundle of namespace, quota, RBAC, and network policy objects automatically:
apiVersion: platform.company.com/v1alpha1
kind: Tenant
metadata:
name: acme
spec:
admins:
- group: acme-admins
resourceTier: standard
environments:
- production
- staging
A controller watching Tenant objects can then materialize everything covered earlier in this article — namespaces, quotas, RBAC bindings, default-deny network policies, Kyverno label requirements — from a single, reviewable, self-service request. This is genuinely where multi-tenant platforms need to land eventually if onboarding volume grows past a handful of teams; hand-applying YAML bundles per tenant doesn’t scale past a certain point, both operationally and in terms of consistency drift between tenants onboarded at different times by different people.
Common Mistakes
- Relying on namespaces alone as a security boundary for genuinely untrusted tenants — namespace isolation doesn’t protect against kernel-level or node-level attacks.
- Forgetting NetworkPolicy enforcement requires a compatible CNI — creating NetworkPolicy manifests that silently do nothing because the CNI doesn’t enforce them.
- Not setting ResourceQuotas, letting one noisy tenant starve others of cluster capacity.
- Giving tenants
cluster-adminor overly broad ClusterRole bindings out of convenience during initial rollout, then struggling to claw back permissions later.
Best Practices
- Match your isolation depth to your actual trust model — don’t over-engineer cluster-per-tenant for internal, trusted teams, but don’t under-engineer shared clusters for genuinely untrusted external workloads either.
- Automate tenant onboarding (namespace + RBAC + quota + network policy + labels) as a single reviewed, versioned bundle rather than manual, ad hoc steps.
- Enforce labeling conventions via policy engine, not documentation alone — documentation gets ignored, admission control doesn’t.
- Revisit and right-size ResourceQuotas periodically based on actual usage, not just initial guesses.
Summary
Kubernetes multi-tenancy is built from layers, not a single switch: namespaces for logical separation, RBAC for access control, ResourceQuotas and LimitRanges for fair resource sharing, NetworkPolicies for traffic isolation, and — for stronger guarantees — dedicated node groups, Fargate, or fully separate clusters. On EKS specifically, pairing namespace-per-tenant patterns with Karpenter NodePools or Fargate profiles gives you a scalable middle ground between “one cluster for everyone” and “one cluster per team.” Choose your isolation depth deliberately based on whether your tenants are trusted internal teams or genuinely untrusted external workloads — that decision shapes almost everything else in the design.
