Mission 1.4 · Stage 1 — GitOps with Argo CD
Environments, Overlays, and the CI → Git Handoff
Goal: Dev and prod from one base using Kustomize overlays, a Helm-sourced dependency, CI that bumps image tags by commit, and an app-of-apps root.
By the end of this mission
- Structure a config repo with a base and per-environment overlays
- Deploy a Helm chart through Argo CD with values in Git
- Have CI update the image tag in the config repo instead of deploying
- Promote dev → prod with a pull request
- Bootstrap everything from one root Application (app of apps) and restrict it with an AppProject
Part 1
Understand it first
Base and overlays
Copy-pasting manifests per environment drifts quickly. KUSTOMIZE keeps one base/ and small overlays/<env>/ that patch only what differs (replicas, resources, image tag, hostnames). HELM does the same with one chart and a values file per environment (Kubernetes course, Helm topic). Both work in Argo CD. Common choice: Helm for third-party software (you consume someone's chart), Kustomize for your own apps. You can also combine them.
Environments as FOLDERS on one branch (overlays/dev, overlays/prod) are the recommended layout. Environments as BRANCHES (dev, prod branches) sound natural but make promotion a merge between long-lived branches full of conflicts and unrelated changes.
The CI → Git handoff
The app repo's pipeline (CI/CD course) still builds, tests, scans, and pushes the image, tagged with the Git SHA, never latest. Its last step changes: instead of kubectl apply, it commits the new tag into the config repo's dev overlay (kustomize edit set image). Argo deploys it. Promotion to prod is a PR that copies the same tag into the prod overlay, so the image that was tested in dev is byte-for-byte the one that ships.
Argo CD Image Updater can automate the dev bump by watching the registry, but an explicit CI commit is simpler to reason about and to audit.
App of apps and AppProjects
Applying Application YAML by hand doesn't scale. In the APP OF APPS pattern, one root Application points at the argocd/ folder of Applications, so adding a new app is just adding a file. A cluster can be rebuilt by installing Argo CD and applying one root Application.
An APPPROJECT fences Applications: which repos they may use, which clusters/namespaces they may deploy to, and which kinds they may create (for example, no ClusterRoles for app teams). It's Argo CD's multi-tenancy boundary, used with its RBAC and SSO groups.
Part 2
Your project after this mission · 11 files change
- shoplite-gitops/
- apps/
- web/
- base/
- deployment.yamlnew
- kustomization.yamlnew
- service.yamlnew
- overlays/
- dev/
- kustomization.yamlnew
- prod/
- kustomization.yamlnew
- argocd/
- apps/
- redis-dev.yamlnew
- web-dev.yamlmodified
- web-prod.yamlnew
- projects/
- shoplite.yamlnew
- root.yamlnew
- shoplite-web/
- .github/
- workflows/
- release.ymlmodified
Part 3
Build it, step by step
- 1
Move manifests into a base
Move deployment.yaml, service.yaml, and the migrate Job into
apps/web/base/and list them in a kustomization.shoplite-gitops/apps/web/base/kustomization.yamlwhole fileyaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - deployment.yaml - service.yaml - migrate-job.yaml labels: - pairs: { app.kubernetes.io/part-of: shoplite } - 2
The dev overlay
The
imagesblock is what CI will edit.newTagpins an immutable tag.shoplite-gitops/apps/web/overlays/dev/kustomization.yamlwhole fileyaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: shoplite-dev resources: [../../base] images: - name: nginxdemos/hello newTag: "0.4" patches: - target: { kind: Deployment, name: web } patch: | - op: add path: /spec/replicas value: 1 - 3
The prod overlay
More replicas, bigger requests, a PodDisruptionBudget would go here too. Prod lags dev by one version until it's promoted.
shoplite-gitops/apps/web/overlays/prod/kustomization.yamlwhole fileyaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: shoplite-prod resources: [../../base] images: - name: nginxdemos/hello newTag: "0.3" patches: - target: { kind: Deployment, name: web } patch: | - op: add path: /spec/replicas value: 3 - op: replace path: /spec/template/spec/containers/0/resources/requests/cpu value: 250mterminal$ kustomize build apps/web/overlays/prod | grep -E 'namespace|replicas|image:|cpu'── expected output ──namespace: shoplite-prodreplicas: 3image: nginxdemos/hello:0.3cpu: 250m - 4
Fence it with an AppProject
ShopLite apps may only come from this repo and the Bitnami chart registry, only deploy to
shoplite-*namespaces, and can't create cluster-scoped resources except Namespaces.shoplite-gitops/argocd/projects/shoplite.yamlwhole fileyaml apiVersion: argoproj.io/v1alpha1 kind: AppProject metadata: name: shoplite namespace: argocd spec: description: ShopLite workloads sourceRepos: - https://github.com/<you>/shoplite-gitops.git - registry-1.docker.io/bitnamicharts destinations: - server: https://kubernetes.default.svc namespace: shoplite-* clusterResourceWhitelist: - group: "" kind: Namespace - 5
One Application per environment
Move the Applications into
argocd/apps/.web-devnow points atapps/web/overlays/devwithproject: shoplite.web-prodis the same with the prod overlay. Prod keepsselfHealbut you may leavepruneoff there at first, which is a common, cautious choice.shoplite-gitops/argocd/apps/web-prod.yamlwhole fileyaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: web-prod namespace: argocd spec: project: shoplite source: repoURL: https://github.com/<you>/shoplite-gitops.git targetRevision: main path: apps/web/overlays/prod destination: server: https://kubernetes.default.svc namespace: shoplite-prod syncPolicy: automated: { prune: false, selfHeal: true } syncOptions: [CreateNamespace=true] - 6
A third-party Helm chart, values in Git
ShopLite's cart cache uses Redis. Argo renders the chart itself (
helm template), so there's no Tiller and nohelm installstate: the Helm release doesn't show inhelm list, and Argo tracks the resources instead. PintargetRevisionto an exact chart version. The Stateful Systems course covers running Redis properly.shoplite-gitops/argocd/apps/redis-dev.yamlwhole fileyaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: redis-dev namespace: argocd spec: project: shoplite source: repoURL: registry-1.docker.io/bitnamicharts chart: redis targetRevision: 22.0.7 helm: valuesObject: architecture: standalone auth: { enabled: true, existingSecret: redis-auth } master: persistence: { size: 1Gi } destination: server: https://kubernetes.default.svc namespace: shoplite-dev syncPolicy: automated: { prune: true, selfHeal: true } - 7
The root Application (app of apps)
The root watches
argocd/recursively: projects and apps. From now on, the only thing you everkubectl applyis this file, once per cluster. Delete the old Applications you applied by hand; the root recreates them from Git.shoplite-gitops/argocd/root.yamlwhole fileyaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: root namespace: argocd spec: project: default source: repoURL: https://github.com/<you>/shoplite-gitops.git targetRevision: main path: argocd directory: recurse: true exclude: root.yaml destination: server: https://kubernetes.default.svc namespace: argocd syncPolicy: automated: { prune: true, selfHeal: true }terminal$ git add . && git commit -m 'kustomize overlays, project, app-of-apps' && git pushkubectl apply -f argocd/root.yamlargocd app list── expected output ──NAME CLUSTER NAMESPACE PROJECT STATUS HEALTHroot https://kubernetes.default.svc argocd default Synced Healthyredis-dev https://kubernetes.default.svc shoplite-dev shoplite Synced Healthyweb-dev https://kubernetes.default.svc shoplite-dev shoplite Synced Healthyweb-prod https://kubernetes.default.svc shoplite-prod shoplite Synced Healthy - 8
CI commits the tag instead of deploying
The last job of the app repo's release workflow (CI/CD course, GitHub Actions). It uses a deploy key or GitHub App token with write access to ONLY the config repo, so there are no cluster credentials anywhere in CI. The commit message links back to the source commit for traceability.
shoplite-web/.github/workflows/release.ymladd to fileyaml bump-dev: needs: build-and-push # image shoplite-web:sha-${{ github.sha }} already pushed + signed runs-on: ubuntu-latest steps: - uses: actions/checkout@v5 with: repository: <you>/shoplite-gitops token: ${{ secrets.GITOPS_REPO_TOKEN }} - uses: imranismail/setup-kustomize@v2 - name: Set dev image tag run: | cd apps/web/overlays/dev kustomize edit set image nginxdemos/hello=ghcr.io/<you>/shoplite-web:sha-${{ github.sha }} git config user.name "shoplite-ci" git config user.email "ci@shoplite.dev" git commit -am "web(dev): deploy ${{ github.sha }} from ${{ github.repository }}" git push - 9
Promote to prod with a PR
Promotion copies the tested tag from dev to prod. A branch-protection rule on
mainrequires an approval for changes underapps/*/overlays/prod/(CODEOWNERS), so deploys to prod get the same review as code (Git course, branch protection).terminal$ git switch -c promote-web-0.4cd apps/web/overlays/prod && kustomize edit set image nginxdemos/hello=nginxdemos/hello:0.4 && cd -git commit -am 'web(prod): promote 0.4' && git push -u origin promote-web-0.4gh pr create --fill --reviewer shoplite/platform── expected output ──https://github.com/<you>/shoplite-gitops/pull/12
Checkpoint — you should now have
- ✓
argocd app listshows root, web-dev, web-prod, and redis-dev, all Synced and Healthy. - ✓Dev runs 0.4 and prod runs 0.3 until the promotion PR merges.
- ✓CI has no kubeconfig; it can only write to the config repo.
- ✓Deleting
web-devin the UI brings it back within seconds, because the root recreates it.
Part 4
Break it on purpose
Make each change, run the command, and read the error before revealing the diagnosis. Recognising these messages on sight is what makes you fast on a real team. Undo the change afterwards.
Break #1
Deploy outside the AppProject's fence
Change web-prod's destination namespace to kube-system and push.
Break #2
CI loop: the config repo triggers the app build
Put the Kubernetes manifests in the APP repo and have CI commit the new tag to that same repo on every build.
Part 5
Interview questions from this mission
How do you handle multiple environments in GitOps?
What is the app-of-apps pattern?
Kustomize or Helm with Argo CD?
Before you stop