System 12.28 — Test Execution Platform
In one line
CI test runners as a service: job scheduling, resource pools, parallelism, artifact collection.
Think of it like this
An assembly line for running a company's automated tests, farming thousands of tests out to many machines in parallel so a full test suite finishes in minutes instead of hours.
Key ideas
- 01
The platform: a queue of test jobs (per commit/branch) → executor pool (autoscaling VMs/containers) → results/artifacts.
- 02
Sharding parallel tests: split a suite across N executors by file/weight; flaky-test attribution via retry + quarantine.
- 03
Resource models: mac/linux/windows pools, GPU, browser matrix; capacity is the cost problem.
- 04
Scheduling: priority queues (CI vs nightly), preemption, affinity (cache reuse on the same executor).
- 05
Artifacts: test logs, screenshots, coverage must be indexed (object store + metadata) and TTL'd.
- 06
The 'production-heavy' angle: it's a job scheduler (12.24) with resource pools and flaky-test politics.
- 07
Interviews: mapping a test platform to scheduler + pool + artifact store is the discipline being tested.
Code & diagrams
A job scheduler (System 12.24) wearing a CI hat — the interesting part is sharding one big suite across many machines.
Explain without notes
Why must flaky tests be quarantined rather than endlessly retried — what does retry-without-data do to confidence?
Practice
Design the executor pool auto-scaling and the test-sharding algorithm by historical duration.
Trade-offs
- ↔
Speed = parallel cost; every parallel executor is price — weight sharding balances coverage vs bill.
Completion checklist
I can present test platform = job scheduler + resource pools + sharding + artifact store.