Controlling AWS EKS Access Using AWS IAM And Kubernetes RBAC

Controlling AWS EKS Access Using AWS IAM And Kubernetes RBAC
In this article, we'll explore how we can use an AWS IAM user and grant it permissions inside a k8s cluster on AWS EKS. I'll assume you have an EKS cluster already created.
In AWS EKS, the IAM user used to create AWS EKS is the default k8s admin user and we can also declare other new users/roles via aws-auth ConfigMap.
Adding new users in AWS EKS has two parts
- AWS IAM
- RBAC (Role-Based Access Management on Kubernetes)
In this article, we will add two users
- One with administrator permission for management - k8sadmin
- One to manage only deployments/replicasets/pods to a particular namespace
We can grant IAM users K8s permissions without attaching any AWS permission. Here's how.
Create the AWS IAM K8s Admin User
This IAM user will not need AWS console access, but programmatic access will be necessary. No need to set any permissions for AWS account and tags. Let's create the k8sadmin
user and save the credentials(AWS access and secrets key) and IAM arn, we'll need them soon.
Use this AWS user guide for help on creating this user.
Now we need to
1. Edit the aws-auth configmap.
1 $ kubectl edit configmap aws-auth -n kube-system
2. Map the IAM user k8sadmin
arn to a pre-defined systems:masters
group. That group gives admin rights on the k8s cluster to the new IAM user.
1 data:
2 mapRoles: |
3 - groups:
4 - system:bootstrappers
5 - system:nodes
6 rolearn: arn:aws:iam::<accountid>:role/<node-role>
7 username: system:node:{{EC2PrivateDNSName}}
8 mapUsers: |
9 - userarn: arn:aws:iam::<accountid>:user/k8sadmin
10 username: k8sadmin
11 groups:
12 - system:masters
13 kind: ConfigMap
14 metadata:
15 creationTimestamp: "2021-07-08T09:47:15Z"
16 name: aws-auth
17 namespace: kube-system
18 resourceVersion: "2315"
19 selfLink: /api/v1/namespaces/kube-system/configmaps/aws-auth
20 uid: 12c84ad916b6-5df31bad-a165-11e9-90f3
3. To test new IAM user run the aws configure --profile k8sadmin
command and configure the IAM user credentials for the key and secret key.
4. Export the environmental varaible AWS_DEFAULT_PROFILE to the new IAM user with the following command: export AWS_DEFAULT_PROFILE=k8sadmin
5. Check the IAM user in use with the aws sts get-caller-identity
command:
1 $ aws sts get-caller-identity
2 {
3 "UserId": "AIDAVF3YR75UYCMUXIBIA",
4 "Account": "<accountid>",
5 "Arn": "arn:aws:iam::<accountid>:user/k8sadmin"
6 }
6. Validate the k8sadmin
user has admin rights with the following kubectl commands. If all the kubectl commands return real data, our k8s is using the new user.
1$ kubectl get nodes
2 NAME STATUS ROLES AGE VERSION
3 ip-**-**-*-***.ap-south-1.compute.internal Ready <none> 99d v1.18.9-eks-d1db3c
4 ip-**-**-*-***.ap-south-1.compute.internal Ready <none> 113d v1.18.9-eks-d1db3c
5$ kubectl -n kube-system get pod
6 NAME READY STATUS RESTARTS AGE
7 aws-node-q6xgv 1/1 Running 0 51m
8 aws-node-h9qw4 1/1 Running 0 51m
9 coredns-8f33c5c4b9-5j9ck 1/1 Running 0 21m
10 coredns-9f21c6c4b9-sl9t5 1/1 Running 0 21m
11 kube-proxy-ffrkb 1/1 Running 0 51m
12 kube-proxy-tnr6r 1/1 Running 0 51m
Now we have an IAM user with no permissions in our AWS account, but with admin rights in our Kubernetes cluster
Create Deployment Only User
Our administrative rights are ready to go, but we need to establish our user which only has deployment permissions in the development namespace. We will use our k8sadmin
administrative user to create this limited user.
Let's start by creating the namespace.
1$ kubectl create namespace development
We will use the reference from the below article to create the k8s role and role-binding.
Bitmani Configure RBAC in your Kubernetes Cluster
1. Create the role for managing deployments in a new file called role-deployment-manager.yaml
. In this role YAML, we are adding the rules that allow a user to execute several operations(get/list/watch/create/update/patch/delete) on Deployments, Pods, and ReplicaSets.
1kind: Role
2 apiVersion: rbac.authorization.k8s.io/v1beta1
3 metadata:
4 namespace: development
5 name: development-manager
6 rules:
7 - apiGroups: ["", "extensions", "apps"]
8 resources: ["deployments", "replicasets", "pods"]
9 verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
2. Create the role in the above YAML
1$ kubectl apply -f role-deployment-manager.yaml
2 role.rbac.authorization.k8s.io/deployment-manager created
3. Bind the role to the developer user by creating this file: rolebinding-deployment-manager.yaml
:
1 kind: RoleBinding
2 apiVersion: rbac.authorization.k8s.io/v1beta1
3 metadata:
4 name: development-manager-binding
5 namespace: development
6 subjects:
7 - kind: User
8 name: developer
9 apiGroup: ""
10 roleRef:
11 kind: Role
12 name: development-manager
13 apiGroup: ""
4. Create the above role-binding with kubectl:
1$ kubectl apply -f rolebinding-deployment-manager.yaml
2 rolebinding.rbac.authorization.k8s.io/deployment-manager-binding created
Now Let's create an IAM user named developer
in the AWS console (as above, only programmatic access, no AWS account permissions, no tags, save credentials, and IAM arn)
After creating developer
IAM, edit the aws-auth configmap (kubectl edit configmap aws-auth -n kube-system
) and add a new user mapping for the developer IAM user, but this time mapped to the development-manager
role.
1 data:
2 mapRoles: |
3 - groups:
4 - system:bootstrappers
5 - system:nodes
6 rolearn: arn:aws:iam::<accountid>:role/<node-role>
7 username: system:node:{{EC2PrivateDNSName}}
8 mapUsers: |
9 - userarn: arn:aws:iam::<accountid>:user/k8sadmin
10 username: k8sadmin
11 groups:
12 - system:masters
13 - userarn: arn:aws:iam::<accountid>:user/developer
14 username: developer
15 groups:
16 - development-manager
17 kind: ConfigMap
18 metadata:
19 creationTimestamp: "2021-07-08T09:47:15Z"
20 name: aws-auth
21 namespace: kube-system
22 resourceVersion: "2315"
23 selfLink: /api/v1/namespaces/kube-system/configmaps/aws-auth
24 uid: 12c84ad916b6-5df31bad-a165-11e9-90f3
To test the developer
IAM user permission, we need to create the aws profile for the developer IAM user and configure the AWS key and secrets key and export the new profile.
1$ aws configure - profile developer
2$ export AWS_DEFAULT_PROFILE="developer"
Now let's test out the developer k8s user permissions by trying to do something we shouldn't be able to
1$ kubectl get node
2Error from server (Forbidden): nodes is forbidden: User "developer" cannot list resource "nodes" in API group "" at the cluster scope
And we're blocked - perfect, this is the expected output.
Now lets do something we should be able to do, deploy Nginx in the development
namespace.
1$ kubectl apply -f https://k8s.io/examples/application/deployment.yaml -n development
2deployment.apps/nginx-deployment created
Summary
In this article, we covered how to create an AWS IAM user with no AWS permissions. We then granted that IAM user administrative permissions within our AWS EKS cluster, and connected together with the IAM access key and secret.
We also created a limited deployment
user which our Dev team (or CI/CD!) can use to deploy their application. We tested to make sure this user didn't have any additional rights.