Mission 1.3 · Stage 1 — GitOps with Argo CD
Automated Sync, Self-Heal, Pruning, and Sync Waves
Goal: Argo CD deploys every merge automatically, reverts manual cluster edits, deletes what's removed from Git, and applies resources in the right order.
By the end of this mission
- Turn on automated sync with prune and selfHeal, and know the risks of each
- Watch Argo detect and undo drift
- Ignore fields that other controllers legitimately own (HPA replicas)
- Order resources with sync waves and run migrations with hooks
Part 1
Understand it first
Three switches, three behaviours
automated: {}: sync whenever Git changes. prune: true: delete live resources that were removed from Git (off by default, because deleting is the dangerous direction). selfHeal: true: when the LIVE state changes (someone ran kubectl edit), put it back to match Git, usually within seconds.
With all three on, Git is the only way to change the cluster. That's the goal, but it has consequences: an on-call engineer's emergency kubectl scale will be reverted. The emergency path must be 'commit to Git' (fast-tracked review) or, in rare cases, temporarily disabling auto-sync on that app. Write that into your runbook (SRE course, incident response).
Drift and ignoreDifferences
DRIFT is any difference between live and desired state. Some differences are legitimate: an HPA changes spec.replicas, a mutating webhook injects a sidecar, a controller adds default fields. ignoreDifferences tells Argo to leave specific fields alone; otherwise it fights the HPA forever. Simplest rule for HPAs: don't set replicas in Git at all.
Sync waves and hooks
By default Argo applies resources in a sensible kind order (Namespaces, then ConfigMaps/Secrets, then Deployments). SYNC WAVES (argocd.argoproj.io/sync-wave: "-1") give explicit ordering: lower waves apply first, and each wave must be Healthy before the next starts. HOOKS are resources (usually Jobs) that run at a phase: PreSync (database migrations), Sync, PostSync (smoke tests, notifications), SyncFail (cleanup). A failing PreSync migration stops the rollout before new code runs against an old schema.
Part 2
Your project after this mission · 4 files change
- shoplite-gitops/
- apps/
- web/
- deployment.yamlmodified
- hpa.yamlnew
- migrate-job.yamlnew
- argocd/
- web-dev.yamlmodified
Part 3
Build it, step by step
- 1
Turn on automation
retryhandles transient failures (a CRD not yet ready).revisionHistoryLimitkeeps the history list short.shoplite-gitops/argocd/web-dev.yamlwhole fileyaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: web-dev namespace: argocd spec: project: default source: repoURL: https://github.com/<you>/shoplite-gitops.git targetRevision: main path: apps/web destination: server: https://kubernetes.default.svc namespace: shoplite-dev revisionHistoryLimit: 10 syncPolicy: automated: prune: true selfHeal: true retry: limit: 3 backoff: { duration: 10s, factor: 2, maxDuration: 2m } syncOptions: - CreateNamespace=true ignoreDifferences: - group: apps kind: Deployment jsonPointers: [/spec/replicas] - 2
Watch self-heal undo a manual change
Scale by hand and watch. Within seconds Argo notices live ≠ Git and restores it. Here the replicas field is ignored, so try an image change instead, which is exactly the kind of 'quick fix' GitOps forbids.
terminal$ kubectl apply -f argocd/web-dev.yamlkubectl -n shoplite-dev set image deploy/web web=nginxdemos/hello:latestsleep 10; kubectl -n shoplite-dev get deploy web -o jsonpath='{.spec.template.spec.containers[0].image}'argocd app history web-dev | tail -1── expected output ──deployment.apps/web image updatednginxdemos/hello:0.32 2026-09-27 10:31:02 +0530 IST main (d4e5f6a) - 3
Let the HPA own replicas
Remove
replicas: 2from the Deployment and add an HPA (Kubernetes course, autoscaling). With the ignoreDifferences entry, Argo stays Synced while the HPA scales.shoplite-gitops/apps/web/hpa.yamlwhole fileyaml apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: web spec: scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: web } minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: { name: cpu, target: { type: Utilization, averageUtilization: 70 } } - 4
Run a migration before the new version starts
A PreSync hook Job runs before anything else in the sync.
hook-delete-policy: BeforeHookCreationdeletes the previous run's Job so the name can be reused. Migrations must be backward compatible (expand, then contract), because old pods keep running during the rollout.shoplite-gitops/apps/web/migrate-job.yamlwhole fileyaml apiVersion: batch/v1 kind: Job metadata: name: db-migrate annotations: argocd.argoproj.io/hook: PreSync argocd.argoproj.io/hook-delete-policy: BeforeHookCreation spec: backoffLimit: 1 template: spec: restartPolicy: Never containers: - name: migrate image: busybox:1.37 command: ["sh", "-c", "echo running migrations for 0.3 && sleep 5"]terminal$ git add . && git commit -m 'web: hpa + presync migration' && git pushargocd app wait web-dev --sync --health && argocd app get web-dev | grep -A6 HOOK── expected output ──GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGEbatch Job shoplite-dev db-migrate Succeeded PreSync job.batch/db-migrate created... - 5
Prune: remove from Git, removed from cluster
Delete the HPA file, push, and it disappears from the cluster. Without
prune: trueit would stay forever as an orphan, marked OutOfSync with a 'requires pruning' flag.terminal$ git rm apps/web/hpa.yaml && git commit -m 'web: drop hpa' && git pushargocd app wait web-dev --sync && kubectl -n shoplite-dev get hpa── expected output ──No resources found in shoplite-dev namespace.
Checkpoint — you should now have
- ✓A manual
kubectl set imageis reverted within seconds. - ✓The PreSync migration Job runs before each sync.
- ✓Removing a file from Git removes the resource from the cluster.
Part 4
Break it on purpose
Make each change, run the command, and read the error before revealing the diagnosis. Recognising these messages on sight is what makes you fast on a real team. Undo the change afterwards.
Break #1
Fight the autoscaler
Put replicas: 2 back in the Deployment and remove the ignoreDifferences block, with the HPA present and some CPU load.
Break #2
A failing migration
Change the migrate command to exit 1 and push.
Part 5
Interview questions from this mission
What do prune and selfHeal do, and why are they off by default?
How do you run database migrations in a GitOps workflow?