Guardrails as Code: Kyverno Policies

1. 1

   Install Kyverno

   Chart kyverno from https://kyverno.github.io/kyverno/, 3 admission controller replicas for HA (if Kyverno is down and its webhook fails closed, nothing can be created). Policies go in a separate Application synced after Kyverno is healthy (sync waves).

   apiVersion: argoproj.io/v1alpha1
   kind: Application
   metadata:
     name: kyverno
     namespace: argocd
     annotations: { argocd.argoproj.io/sync-wave: "-5" }
   spec:
     project: default
     source:
       repoURL: https://kyverno.github.io/kyverno/
       chart: kyverno
       targetRevision: 3.5.1
       helm:
         valuesObject:
           admissionController: { replicas: 3 }
     destination:
       server: https://kubernetes.default.svc
       namespace: kyverno
     syncPolicy:
       automated: { prune: true, selfHeal: true }
       syncOptions: [CreateNamespace=true, ServerSideApply=true]

2. 2

   Validate: require requests and limits

   Every container needs CPU/memory requests and a memory limit, the input Karpenter and the scheduler rely on. Audit mode first.

   apiVersion: kyverno.io/v1
   kind: ClusterPolicy
   metadata:
     name: require-requests-limits
     annotations:
       policies.kyverno.io/description: Containers must set CPU/memory requests and a memory limit.
   spec:
     validationFailureAction: Audit      # → Enforce after teams comply
     background: true
     rules:
       - name: check-resources
         match:
           any: [{ resources: { kinds: [Pod] } }]
         exclude:
           any: [{ resources: { namespaces: [kube-system, kyverno, argocd] } }]
         validate:
           message: "Set resources.requests.cpu, requests.memory and limits.memory on every container."
           pattern:
             spec:
               containers:
                 - resources:
                     requests: { cpu: "?*", memory: "?*" }
                     limits: { memory: "?*" }

3. 3

   Validate: only our registries

   Images must come from the company registry (ECR) or a small allow-list. This blocks typo-squatted and unreviewed public images.

   apiVersion: kyverno.io/v1
   kind: ClusterPolicy
   metadata: { name: restrict-image-registries }
   spec:
     validationFailureAction: Enforce
     background: true
     rules:
       - name: allowed-registries
         match:
           any: [{ resources: { kinds: [Pod], namespaceSelector: { matchLabels: { shoplite.dev/team: "?*" } } } }]
         validate:
           message: "Images must come from 123456789012.dkr.ecr.ap-south-1.amazonaws.com or public.ecr.aws/shoplite."
           pattern:
             spec:
               containers:
                 - image: "123456789012.dkr.ecr.ap-south-1.amazonaws.com/* | public.ecr.aws/shoplite/*"

4. 4

   Mutate: add defaults

   Add a restrictive pod securityContext (runAsNonRoot, the default seccomp profile) when a team hasn't set one. The +( ) anchors only add a field if it's missing, so teams can still override. Mutations make compliance free for teams.

   apiVersion: kyverno.io/v1
   kind: ClusterPolicy
   metadata: { name: add-defaults }
   spec:
     rules:
       - name: default-security-context
         match:
           any: [{ resources: { kinds: [Pod] } }]
         exclude:
           any: [{ resources: { namespaces: [kube-system] } }]
         mutate:
           patchStrategicMerge:
             spec:
               securityContext:
                 +(runAsNonRoot): true
                 +(seccompProfile): { type: RuntimeDefault }

5. 5

   Generate: every team namespace gets defaults

   When a namespace labelled shoplite.dev/team is created, Kyverno generates a default-deny NetworkPolicy (Kubernetes course, NetworkPolicy) and a ResourceQuota, and keeps them in sync.

   apiVersion: kyverno.io/v1
   kind: ClusterPolicy
   metadata: { name: namespace-defaults }
   spec:
     rules:
       - name: default-deny-ingress
         match:
           any: [{ resources: { kinds: [Namespace], selector: { matchLabels: { shoplite.dev/team: "?*" } } } }]
         generate:
           apiVersion: networking.k8s.io/v1
           kind: NetworkPolicy
           name: default-deny-ingress
           namespace: "{{request.object.metadata.name}}"
           synchronize: true
           data:
             spec:
               podSelector: {}
               policyTypes: [Ingress]
       - name: quota
         match:
           any: [{ resources: { kinds: [Namespace], selector: { matchLabels: { shoplite.dev/team: "?*" } } } }]
         generate:
           apiVersion: v1
           kind: ResourceQuota
           name: team-quota
           namespace: "{{request.object.metadata.name}}"
           synchronize: true
           data:
             spec:
               hard: { requests.cpu: "20", requests.memory: 40Gi, pods: "100" }

6. 6

   Test policies in CI

   The Kyverno CLI applies policies to sample manifests offline. Put this in the platform repo's pipeline so a policy change can't silently block everything.

   apiVersion: cli.kyverno.io/v1alpha1
   kind: Test
   metadata: { name: platform-policies }
   policies: [../policies/require-requests-limits.yaml, ../policies/restrict-registries.yaml]
   resources: [good-pod.yaml, bad-pod.yaml]
   results:
     - { policy: require-requests-limits, rule: check-resources, resources: [good-pod], result: pass }
     - { policy: require-requests-limits, rule: check-resources, resources: [bad-pod], result: fail }

   terminalcopy cmd

   $ kyverno test shoplite-platform/addons/kyverno/tests/

   kubectl get policyreport -A -o wide | head -4

   ── expected output ──

   Test Summary: 2 tests passed and 0 tests failed

   NAMESPACE NAME KIND NAME PASS FAIL WARN ERROR SKIP

   catalog 3f1c... Deployment catalog 4 1 0 0 0

   checkout 9b2e... Deployment checkout 5 0 0 0 0

Make each change, run the command, and read the error before revealing the diagnosis. Recognising these messages on sight is what makes you fast on a real team. Undo the change afterwards.