As your team and projects grow, managing a single EKS cluster can get complicated. You'll likely have different teams like a "Backend Team" and a "Frontend Team" all needing to deploy their applications. If everyone deploys to the same place, things can get messy. Resources can be over-consumed, security becomes a concern, and it's hard to tell who owns what.
The solution to this common problem is multi-tenancy. Multi-tenancy is the practice of having multiple tenants (in this case, teams or applications) share the same EKS cluster while being logically isolated from each other. This is a highly efficient and cost-effective way to manage your Kubernetes infrastructure.
This blog post will walk you through a simple yet powerful way to achieve multi-tenancy on AWS EKS using two fundamental concepts: Kubernetes Namespaces and AWS IAM for EKS. By the end, you will have a practical setup that provides clear separation and security for different teams.
The Problem: A Single, Shared Cluster
Imagine a scenario where both the Backend Team and Frontend Team deploy their applications to the default namespace.
Chaos: Deployments might accidentally have the same names, leading to conflicts.
Resource Hogs: The Frontend Team's application could suddenly get a lot of traffic and use up all the cluster's CPU, making the Backend Team's services slow or unresponsive.
Security Risks: An engineer from the Backend Team might have permissions to accidentally delete a critical service belonging to the Frontend Team.
No Ownership: When something goes wrong, it's difficult to quickly figure out which team is responsible.
The Solution: Namespaces + IAM
Our solution is to create a dedicated namespace for each team. A namespace is a virtual partition inside a Kubernetes cluster. It gives you a way to divide cluster resources and provide a scope for names. But just creating a namespace isn't enough; we need to enforce who can access what. This is where AWS IAM comes in.
AWS EKS has a powerful feature that lets you map AWS IAM roles directly to Kubernetes RBAC (Role-Based Access Control) permissions. This allows you to say, "The IAM role for the Backend Team's developers can only access the backend-team namespace, and nothing else."
Here's the plan:
Create an EKS Cluster: We'll start with a standard EKS cluster.
Create Namespaces: We'll create two namespaces, backend-team and frontend-team.
Create IAM Roles: We'll set up two IAM roles, BackendTeamDeveloper and FrontendTeamDeveloper.
Map IAM to RBAC: We'll use EKS's built-in access management to grant each IAM role specific permissions for their respective namespaces.
Deploy and Demo: We'll deploy a simple application for each team and demonstrate how they are isolated.
Practical Demo: Step-by-Step
This demo assumes you have the AWS CLI and kubectl configured.
Step 1: Create an EKS Cluster
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: multi-tenancy-demo
region: us-east-1
version: "1.28"
managedNodeGroups:
- name: standard-nodes
instanceType: t3.medium
desiredCapacity: 2
Now, create the cluster:
eksctl create cluster -f cluster.yaml
Step 2: Create Namespaces
Once your cluster is active, let's create a dedicated namespace for each team.
kubectl create namespace backend-team
kubectl create namespace frontend-team
You can verify the namespaces are created with:
kubectl get namespaces
Step 3: Create IAM Roles
In the AWS Console, navigate to the IAM service and create two IAM roles:
BackendTeamDeveloper
FrontendTeamDeveloper
These roles are what your team members will assume to interact with the cluster. For this demo, you can leave the roles without any specific permissions initially. The EKS access management will handle the permissions for the cluster itself.
Step 4: Map IAM Roles to Namespaces and Kubernetes Role RoleBinding
This is the most critical step for security. We'll use EKS's Access Management feature to map our IAM roles to specific permissions within the cluster.
You need to use the AWS CLI to create an Access Entry and Access Policy for each team. Replace 123456789 with your AWS account ID.
# For the Backend Team
eksctl create accessentry --cluster multi-tenancy-demo \ --principal-arn arn:aws:iam::123456789:role/BackendTeamDeveloper \ --type STANDARD
# For the Frontend Team
eksctl create accessentry --cluster multi-tenancy-demo \ --principal-arn arn:aws:iam::123456789:role/FrontendTeamDeveloper \ --type STANDARD
Now, we'll create the RBAC resources.
# This Role defines permissions for the backend team within their namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: backend-team-developer-role
namespace: backend-team
rules:
- apiGroups: ["", "apps", "extensions"]
resources: ["pods", "deployments", "services", "configmaps"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list"]
---
# This RoleBinding links the IAM principal to the Role
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: backend-team-developer-rolebinding
namespace: backend-team
subjects:
- kind: User
name: arn:aws:iam::123456789:role/BackendTeamDeveloper # This must match the principal ARN
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: backend-team-developer-role
apiGroup: rbac.authorization.k8s.io
# This Role defines permissions for the frontend team within their namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: frontend-team-developer-role
namespace: frontend-team
rules:
- apiGroups: ["", "apps", "extensions"]
resources: ["pods", "deployments", "services", "configmaps"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
# This RoleBinding links the IAM principal to the Role
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: frontend-team-developer-rolebinding
namespace: frontend-team
subjects:
- kind: User
name: arn:aws:iam::123456789:role/FrontendTeamDeveloper # This must match the principal ARN
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: frontend-team-developer-role
apiGroup: rbac.authorization.k8s.io
Apply these files to your cluster:
kubectl apply -f backend-team-rbac.yaml
kubectl apply -f frontend-team-rbac.yaml
Step 5: Demoing the Isolation
Let's see how this works in practice.First, assume the BackendTeamDeveloper IAM role using the AWS CLI.
aws sts assume-role --role-arn arn:aws:iam::123456789012:role/BackendTeamDeveloper --role-session-name BackendTeamSession
Use the returned credentials to configure your AWS CLI session and verify identity
aws sts get-caller-identity
Now, try to deploy an Nginx server into the backend-team namespace.
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend-app
namespace: backend-team
spec:
replicas: 1
selector:
matchLabels:
app: backend-app
template:
metadata:
labels:
app: backend-app
spec:
containers:
- name: nginx
image: nginx:latest
Deploy the application:
kubectl apply -f backend-app.yaml
This should work perfectly. The pod will be created in the backend-team namespace.
Now, with the same IAM role (BackendTeamDeveloper), try to list pods in the frontend-team namespace:
Error from server (Forbidden): pods is forbidden: User "arn:aws:sts::123456789:assumed-role/BackendTeamDeveloper/..." cannot list resource "pods" in API group "" in the namespace "frontend-team"
Again, a Forbidden error confirms our security model is working perfectly.
Conclusion
This hands-on guide confirms that secure multi-tenancy on AWS EKS is built on a two-step process: using eksctl to map IAM roles to Kubernetes identities, then using Kubernetes RBAC to enforce granular permissions. The Forbidden errors we encountered were the ultimate proof of a successful setup, confirming that our security model is fully functional and providing a robust foundation for shared EKS clusters.
