Topic 0.2
The Pipeline Model: Stages, Jobs, and Runners
In one line
Every CI/CD platform — Jenkins, GitHub Actions, GitLab CI — shares the same underlying vocabulary: a pipeline of stages, each containing jobs, each executed by a runner or agent somewhere.
Think of it like this
An assembly line (Docker's own course used this analogy for pipes, and it fits just as well here) — a PIPELINE is the whole line, a STAGE is one station on it (build, then test, then deploy), and a JOB is the specific work done at that station, carried out by a specific worker (a RUNNER or AGENT).
Key ideas
- 01
A PIPELINE is the complete, end-to-end automated process triggered by some event (Phase 1.1 covers triggers in depth) — typically composed of sequential STAGES, each representing a distinct phase of the process (
build,test,deploy), run in order, since later stages usually depend on earlier ones succeeding first. - 02
Within a stage, one or more JOBS actually do the real work — a
teststage might contain separate jobs for unit tests, integration tests, and linting, which can often run in PARALLEL (Phase 1.4 covers this directly) since they're typically independent of each other, even though the stages themselves usually run sequentially. - 03
A RUNNER (GitHub Actions' and GitLab's term) or AGENT (Jenkins' term) is the actual machine — physical, virtual, or a container — that executes a job's commands. Platforms typically offer HOSTED runners (managed by the platform itself, spun up fresh for each job) and SELF-HOSTED runners (your own infrastructure, registered with the platform) — Phase 2.4 covers self-hosted runners specifically.
- 04
A WORKSPACE is the actual filesystem area a job executes in — typically starting fresh (a clean checkout of your repository) for each job on a hosted runner, which is exactly why ARTIFACTS (files produced by one job that a later job needs, like a compiled binary) must be explicitly passed between jobs rather than simply assumed to still be sitting on disk.
- 05
This vocabulary (pipeline → stages → jobs → runners) is genuinely universal across every major CI/CD platform, even though each one uses slightly different specific terms and YAML syntax for expressing it — learning this model ONCE, here, is what makes Phases 2 through 4's platform-specific syntax feel like a straightforward vocabulary swap rather than three entirely separate things to learn from scratch.
Code & diagrams
The same shape, regardless of which specific platform's YAML syntax you're reading.
Explain it without notes
Why do stages typically run sequentially while jobs within a stage often run in parallel?
Why can't a job in the 'deploy' stage simply assume a file produced by a job in the earlier 'build' stage is still sitting on disk?
Practice
Sketch (in plain English or a simple diagram) a pipeline for a hypothetical Java web application: what stages would it have, and what jobs would likely run within each?
For your sketched pipeline, identify which jobs (if any) could genuinely run in parallel within the same stage, and explain why.
Trade-offs
- ↔
Running more jobs in parallel speeds up a pipeline's total wall-clock time, but it also means consuming more compute resources SIMULTANEOUSLY (more runners/agents active at once) — on a platform charging by runner-minutes or with a limited pool of self-hosted agents, there's a genuine, real trade-off between pipeline speed and actual compute cost or capacity, worth being deliberate about rather than parallelizing everything simply because it's possible.
Done when you can
I can define pipeline, stage, job, and runner/agent in my own words.
I understand why stages typically run sequentially while jobs within a stage often run in parallel.
I understand why artifacts must be explicitly passed between jobs rather than assumed to persist on disk.