Kubernetes RBAC

Who's allowed to do what, and how Kubernetes — and EKS on top of it — decides that on every single API call. Track how many knowledge checks below you clear as you go:

Auth Chain: Every Request to the API Server

graph LR
    classDef request fill:#e74c3c,stroke:#c0392b,color:#fff
    classDef authn fill:#3498db,stroke:#2980b9,color:#fff
    classDef authz fill:#9b59b6,stroke:#8e44ad,color:#fff
    classDef admission fill:#e67e22,stroke:#d35400,color:#fff
    classDef ok fill:#2ecc71,stroke:#27ae60,color:#fff
    classDef deny fill:#c0392b,stroke:#922b21,color:#fff
    classDef apf fill:#16a085,stroke:#117a65,color:#fff

    REQ["API Request kubectl get pods Pod calling K8s API"]:::request

    REQ --> AUTHN["Authentication Who are you? client cert, bearer token, OIDC (EKS IAM), ServiceAccount token"]:::authn
    AUTHN -->|"identity established"| AUTHZ["Authorization (RBAC) Are you allowed? check Role/ClusterRole bindings"]:::authz
    AUTHN -->|"unknown identity"| DENY1["401 Unauthorized"]:::deny

    AUTHZ -->|"allowed"| APF["API Priority and Fairness — seat available?"]:::apf
    AUTHZ -->|"no matching rule"| DENY2["403 Forbidden"]:::deny

    APF -->|"seat assigned"| ADMISSION["Admission Controllers Mutate then Validate LimitRanger, PodSecurity, Webhooks"]:::admission
    APF -->|"queue full / no seat"| THROTTLE["429 Too Many Requests"]:::deny

    ADMISSION -->|"passes all webhooks"| PERSIST["Persist to etcd 200 OK"]:::ok
    ADMISSION -->|"webhook rejects"| DENY3["400/403 from webhook"]:::deny

Same chain, one step at a time:

API Priority and Fairness

RBAC only answers "are you allowed?" It says nothing about "does the API server have room to handle this right now?" That's a separate problem, and Kubernetes solves it with a separate mechanism sitting right after authorization and before admission control: API Priority and Fairness (APF).

Without it, one noisy client — a buggy controller stuck in a hot retry loop, a batch job hammering list across every namespace — can consume every available slot on the API server, and every other client, including the scheduler's own requests and kubelet heartbeats, gets starved out along with it. A misbehaving client shouldn't be able to take down cluster-critical control-plane traffic just by being noisy. APF exists to make sure it can't.

PriorityLevelConfiguration and FlowSchema

Like RBAC being Role/ClusterRole (rules) plus RoleBinding/ClusterRoleBinding (who those rules apply to) instead of one monolithic object, APF splits "how much capacity" from "which requests" into two separate API objects:

• PriorityLevelConfiguration — defines a queue, its concurrency share, and whether it's an exempt priority level (never queued or limited — reserved for traffic that must never be throttled, like the observability/health checks the control plane depends on) or a limited priority level (subject to queuing and possible rejection once its share is exhausted).
• FlowSchema — matches incoming requests to a priority level by user, group, resource, and verb, the same way a RoleBinding matches a subject to a Role.

Keeping these separate means one PriorityLevelConfiguration (say, workload-high) can be reused by many FlowSchemas (one matching a specific controller's ServiceAccount, another matching a specific verb+resource pattern) without redefining the concurrency share every time.

apiVersion: flowcontrol.apiserver.k8s.io/v1
kind: PriorityLevelConfiguration
metadata:
  name: workload-high
spec:
  type: Limited
  limited:
    nominalConcurrencyShares: 30   # this class's slice of total server capacity
    limitResponse:
      type: Queue
      queuing:
        queues: 64
        queueLengthLimit: 50       # requests queued per queue before rejecting
        handSize: 6
---
apiVersion: flowcontrol.apiserver.k8s.io/v1
kind: FlowSchema
metadata:
  name: workload-high-controllers
spec:
  priorityLevelConfiguration:
    name: workload-high
  matchingPrecedence: 500
  distinguisherMethod:
    type: ByUser
  rules:
    - subjects:
        - kind: ServiceAccount
          serviceAccount:
            name: my-controller
            namespace: payments
      resourceRules:
        - apiGroups: ["*"]
          resources: ["*"]
          verbs: ["*"]

Seats, not requests-per-second

The unit APF allocates isn't "requests per second" — it's seats. Every in-flight request occupies a seat for as long as it's being processed, and a priority level's concurrency share determines how many seats out of the API server's total capacity that class of traffic gets at any given moment.

This is a fundamentally different model from a flat rate limiter. A rate limiter counts requests over a time window and doesn't care how long any one of them takes. A seat-based model cares about occupancy: a slow list across a huge namespace holds its seat for the whole time it takes to build and stream that response, while a fast get on a single object frees its seat almost immediately. Two requests that count identically against an RPS limit can consume very different amounts of actual capacity under APF, and the seat model accounts for that directly instead of pretending every request costs the same.

What happens when the queue is full

If a limited priority level's queue is already full when a new matching request arrives, APF doesn't drop the connection or make the client wait indefinitely — it returns 429 Too Many Requests with a Retry-After header, telling the client exactly when it's reasonable to try again.

That's a deliberately different signal from the 403 the AuthZ stage returns earlier in the chain:

• 403 Forbidden — the identity is known, but RBAC has no rule permitting this verb+resource. "You're not allowed," and retrying the identical request will never succeed.
• 429 Too Many Requests — RBAC already said yes. The server is protecting its own capacity right now and asking the client to back off. "You're allowed, but not this instant," and retrying later — which any well-behaved client, including client-go's own libraries, already knows how to do — is exactly the expected response.

RBAC Building Blocks

graph TD
    classDef sa fill:#3498db,stroke:#2980b9,color:#fff
    classDef role fill:#9b59b6,stroke:#8e44ad,color:#fff
    classDef binding fill:#e67e22,stroke:#d35400,color:#fff
    classDef action fill:#2ecc71,stroke:#27ae60,color:#fff
    classDef clusterscope fill:#1abc9c,stroke:#16a085,color:#fff

    subgraph Identities["Who (Subject)"]
        SA["ServiceAccount my-app namespace: payments"]:::sa
        USER["User deepanshu (IAM/OIDC)"]:::sa
        GROUP["Group system:masters"]:::sa
    end

    subgraph Permissions["What (Rules)"]
        ROLE["Role namespace-scoped verbs: get,list,watch resources: pods,services"]:::role
        CR["ClusterRole cluster-scoped can reference any namespace or cluster resources like nodes"]:::clusterscope
    end

    subgraph Bindings["Link (Binding)"]
        RB["RoleBinding binds Role or ClusterRole to subjects in ONE namespace"]:::binding
        CRB["ClusterRoleBinding binds ClusterRole to subjects CLUSTER-WIDE"]:::binding
    end

    SA --> RB
    USER --> RB
    ROLE --> RB
    CR --> RB
    CR --> CRB
    SA --> CRB

    RB --> ACTION["Can get/list/watch pods in the payments namespace"]:::action
    CRB --> ACTION2["Can get/list/watch pods in ALL namespaces"]:::action

ServiceAccount

A ServiceAccount is a Kubernetes identity for pods. Every pod runs as a ServiceAccount. If you don't specify one, it runs as the default ServiceAccount in its namespace.

graph TD
    classDef sa fill:#3498db,stroke:#2980b9,color:#fff
    classDef secret fill:#e74c3c,stroke:#c0392b,color:#fff
    classDef pod fill:#2ecc71,stroke:#27ae60,color:#fff
    classDef api fill:#9b59b6,stroke:#8e44ad,color:#fff

    SA["ServiceAccount: my-app namespace: payments"]:::sa
    TOKEN["Projected ServiceAccount Token /var/run/secrets/kubernetes.io/serviceaccount/token auto-mounted into pod expires every 1h (rotated by kubelet)"]:::secret
    POD["Pod: my-app uses ServiceAccount: my-app token auto-mounted"]:::pod
    API["K8s API Server validates token identity: system:serviceaccount:payments:my-app"]:::api

    SA --> TOKEN
    TOKEN --> POD
    POD -->|"Bearer token in API calls"| API

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app
  namespace: payments
  annotations:
    # IRSA: also allows pod to assume AWS IAM role
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789:role/my-app-s3-role
automountServiceAccountToken: true   # default — set false if pod doesn't call K8s API

Role and ClusterRole

Both are collections of rules — the only difference is scope, and it's a hard boundary, not a preference:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-reader
  namespace: payments
rules:
  - apiGroups: [""]            # "" = core API group (pods, services, configmaps)
    resources: ["pods", "pods/log"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]        # apps group (deployments, replicasets)
    resources: ["deployments"]
    verbs: ["get", "list"]

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: node-reader
rules:
  - apiGroups: [""]
    resources: ["nodes"]       # nodes are cluster-scoped, need ClusterRole
    verbs: ["get", "list", "watch"]
  - apiGroups: ["metrics.k8s.io"]
    resources: ["nodes", "pods"]
    verbs: ["get", "list"]

Verbs reference:

RoleBinding and ClusterRoleBinding

# RoleBinding: grant Role to ServiceAccount IN one namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: my-app-pod-reader
  namespace: payments
subjects:
  - kind: ServiceAccount
    name: my-app
    namespace: payments
  - kind: User             # also works for users
    name: deepanshu
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io
---
# ClusterRoleBinding: grant ClusterRole across ALL namespaces
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: monitoring-cluster-reader
subjects:
  - kind: ServiceAccount
    name: prometheus
    namespace: monitoring
roleRef:
  kind: ClusterRole
  name: cluster-reader
  apiGroup: rbac.authorization.k8s.io

Trick: ClusterRole + RoleBinding — you can bind a ClusterRole via a RoleBinding to scope it to one namespace. Useful for reusable role definitions:

# Define once as ClusterRole
kind: ClusterRole
metadata:
  name: app-role    # reusable template

---
# Bind in namespace A
kind: RoleBinding
metadata:
  namespace: payments    # scoped to payments only
roleRef:
  kind: ClusterRole
  name: app-role

---
# Bind in namespace B with same ClusterRole
kind: RoleBinding
metadata:
  namespace: orders      # scoped to orders only
roleRef:
  kind: ClusterRole
  name: app-role

Try It Yourself: A Live RBAC Evaluator (mirrors kubectl auth can-i)

Everything above is the theory. This walks the real chain — subject → binding → role → rule — against whatever bindings you define, and answers allow or deny the same way kubectl auth can-i would, except every step of the decision stays visible instead of collapsing into one yes/no.

A few modeling choices worth stating up front, all straight from the rules above: a RoleBinding always confines whatever it references — even a ClusterRole — to the single namespace the RoleBinding itself lives in; only a ClusterRoleBinding is cluster-wide. Verbs and resources both support the * wildcard, same as real RBAC rules. And subject matching here is literal, not group-resolved — a binding for group platform-team only matches a query that itself asks "can Group platform-team…", the same simplification kubectl auth can-i --as-group= makes you supply explicitly rather than resolving membership for you.

Load a preset below, then try to break it — query a namespace the binding was never scoped to, or a verb it never granted, and watch the evaluator explain the deny instead of just asserting it.

RBAC Patterns

graph TD
    classDef good fill:#2ecc71,stroke:#27ae60,color:#fff
    classDef bad fill:#e74c3c,stroke:#c0392b,color:#fff
    classDef sa fill:#3498db,stroke:#2980b9,color:#fff

    subgraph Good["Good Patterns"]
        G1["Dedicated ServiceAccount per app not sharing default SA"]:::good
        G2["Narrow verbs: get,list,watch not wildcard *"]:::good
        G3["Namespace-scoped Role when possible not ClusterRole unless needed"]:::good
        G4["automountServiceAccountToken: false for pods that don't call K8s API"]:::good
    end

    subgraph Bad["Anti-Patterns"]
        B1["Using default ServiceAccount all apps in namespace share same identity"]:::bad
        B2["ClusterRoleBinding to system:masters grants cluster-admin to everyone"]:::bad
        B3["Wildcard resources: * wildcard verbs: * = full cluster access"]:::bad
    end

Debug RBAC issues:

# Check what a ServiceAccount can do
kubectl auth can-i get pods \
  --as=system:serviceaccount:payments:my-app \
  -n payments

# List all permissions for a ServiceAccount
kubectl auth can-i --list \
  --as=system:serviceaccount:payments:my-app \
  -n payments

# Describe a RoleBinding to see who has what
kubectl describe rolebinding my-app-pod-reader -n payments

# Check if a specific action is allowed
kubectl auth can-i create deployments --as=system:serviceaccount:payments:my-app -n payments

# Who has cluster-admin? (critical audit check)
kubectl get clusterrolebindings -o json | \
  jq '.items[] | select(.roleRef.name=="cluster-admin") | {name: .metadata.name, subjects: .subjects}'

RBAC — Advanced Patterns and EKS

kubectl auth reconcile

kubectl apply on RBAC objects can produce conflicts if a ClusterRole already exists with different rules. kubectl auth reconcile is the safe way to apply RBAC — it adds missing rules without removing existing ones:

# Apply RBAC idempotently (safe for GitOps pipelines)
kubectl auth reconcile -f rbac-manifests/

# Dry-run first
kubectl auth reconcile -f rbac-manifests/ --dry-run=client

Aggregated ClusterRoles

ClusterRoles can be composed from smaller roles using aggregationRule. Any ClusterRole with a matching label automatically has its rules merged in. Used by Kubernetes to build view, edit, admin roles from extension API groups:

# Base aggregated role — collects rules from labeled sub-roles
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: custom-platform-role
aggregationRule:
  clusterRoleSelectors:
    - matchLabels:
        rbac.example.com/aggregate-to-platform: "true"
rules: []  # auto-populated from matching ClusterRoles

---
# Sub-role that gets merged in automatically
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: platform-secrets-reader
  labels:
    rbac.example.com/aggregate-to-platform: "true"  # triggers aggregation
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "list"]

Token projection and bound service account tokens

Modern K8s (1.21+) uses projected tokens — short-lived (1h default), audience-bound, and tied to a specific pod. They replace the old static Secrets-based tokens.

# Explicitly configure projected token (usually auto-injected, but here for clarity)
spec:
  volumes:
    - name: token
      projected:
        sources:
          - serviceAccountToken:
              path: token
              expirationSeconds: 3600      # 1 hour
              audience: "https://kubernetes.default.svc"
  containers:
    - name: app
      volumeMounts:
        - name: token
          mountPath: /var/run/secrets/kubernetes.io/serviceaccount

# Decode the token to inspect claims
kubectl exec <pod> -- cat /var/run/secrets/kubernetes.io/serviceaccount/token | \
  cut -d. -f2 | base64 -d 2>/dev/null | jq .
# {
#   "aud": ["https://kubernetes.default.svc"],
#   "exp": 1700000000,
#   "iat": 1699996400,
#   "iss": "https://kubernetes.default.svc",
#   "kubernetes.io": {
#     "namespace": "payments",
#     "pod": { "name": "my-app-xxx", "uid": "..." },
#     "serviceaccount": { "name": "my-app", "uid": "..." }
#   },
#   "sub": "system:serviceaccount:payments:my-app"
# }

EKS — aws-auth ConfigMap (legacy) vs Access Entries (current)

EKS authenticates using IAM. The mapping from IAM identity → Kubernetes username/groups is configured two ways:

# kubectl edit configmap aws-auth -n kube-system
apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapRoles: |
    # Worker node IAM role → system:nodes group (required for nodes to join)
    - rolearn: arn:aws:iam::123456789:role/eks-node-role
      username: system:node:{{EC2PrivateDNSName}}
      groups:
        - system:bootstrappers
        - system:nodes
    # Human IAM role → Kubernetes RBAC group
    - rolearn: arn:aws:iam::123456789:role/devops-team
      username: devops-{{SessionName}}
      groups:
        - platform-admins   # bind this group to a ClusterRole in RBAC
  mapUsers: |
    # Specific IAM user (avoid when possible — use roles)
    - userarn: arn:aws:iam::123456789:user/alice
      username: alice
      groups:
        - developers

# Create an access entry (replaces aws-auth ConfigMap rows)
aws eks create-access-entry \
  --cluster-name my-cluster \
  --principal-arn arn:aws:iam::123456789:role/devops-team \
  --type STANDARD \
  --kubernetes-groups platform-admins
Associate with an access policy (AWS-managed or custom)
aws eks associate-access-policy 

--cluster-name my-cluster 

--principal-arn arn:aws:iam::123456789:role/devops-team 

--policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy 

--access-scope '{"type": "cluster"}'
List all access entries
aws eks list-access-entries --cluster-name my-cluster

IRSA — IAM Roles for Service Accounts

IRSA lets a Kubernetes ServiceAccount assume an AWS IAM role without static credentials. The pod gets a projected token, exchanges it at the STS OIDC endpoint, and receives temporary AWS credentials.

# 1. Create OIDC provider for your EKS cluster (one-time per cluster)
eksctl utils associate-iam-oidc-provider \
  --cluster my-cluster --approve

# 2. Create IAM role with trust policy scoped to specific ServiceAccount
OIDC_ID=$(aws eks describe-cluster --name my-cluster \
  --query "cluster.identity.oidc.issuer" --output text | cut -d/ -f5)

aws iam create-role \
  --role-name my-app-s3-role \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {"Federated": "arn:aws:iam::123456789:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/'"$OIDC_ID"'"},
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.us-east-1.amazonaws.com/id/'"$OIDC_ID"':aud": "sts.amazonaws.com",
          "oidc.eks.us-east-1.amazonaws.com/id/'"$OIDC_ID"':sub": "system:serviceaccount:payments:my-app"
        }
      }
    }]
  }'

# 3. Annotate the ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app
  namespace: payments
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789:role/my-app-s3-role
    eks.amazonaws.com/token-expiration: "3600"  # optional: shorter token lifetime

# 4. Verify — pod should have these env vars injected by the EKS pod identity webhook:
kubectl exec <pod> -- env | grep AWS
# AWS_ROLE_ARN=arn:aws:iam::123456789:role/my-app-s3-role
# AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
# AWS_DEFAULT_REGION=us-east-1

# If missing: check the pod admission mutation webhook is running
kubectl get mutatingwebhookconfigurations | grep pod-identity

Same setup, walked through as one flow — including the runtime exchange the four commands above are all in service of:

Common RBAC misconfigurations

RBAC audit one-liners

# Find all cluster-admin bindings
kubectl get clusterrolebindings -o json | \
  jq -r '.items[] | select(.roleRef.name=="cluster-admin") |
  "\(.metadata.name): \(.subjects // [] | map(.name) | join(", "))"'

# Find all ServiceAccounts with cluster-wide permissions
kubectl get clusterrolebindings -o json | \
  jq -r '.items[] | .subjects[]? | select(.kind=="ServiceAccount") |
  "\(.namespace)/\(.name)"' | sort -u

# What can a given ServiceAccount do across all namespaces?
for ns in $(kubectl get ns -o jsonpath='{.items[*].metadata.name}'); do
  kubectl auth can-i --list \
    --as=system:serviceaccount:payments:my-app \
    -n $ns 2>/dev/null | grep -v "^Resources" | grep -v "^\*"
done