Command Palette

Search for a command to run...

Hectal
Phase 9: Evaluating RAG
RAG and Vector Search · Exercise 5 of 5 Previous
Advanced

A regression gate with a floor and a maximum drop

Write gate(metrics_old, metrics_new, floors, max_drop). The first three arguments are dicts from metric name to value, and max_drop is a number.

Return a list of failure messages, in the order of the metrics in metrics_old. For each metric, add "<name> is below its floor" when the new value is less than floors[name], and then "<name> dropped too far" when old - new is greater than max_drop. An empty list means the gate passes.

Write exit_code(failures) that returns 0 for an empty list and 1 otherwise.

What Check tests

  1. No change passes
  2. A small drop above the floor passes
  3. A big drop above the floor fails the drop check
  4. Below the floor fails the floor check
  5. Both messages can appear, in order
  6. exit_code is 0 or 1

Stuck? One hint at a time

The solution unlocks after your first Check. Try first: that is where the learning happens.

main.py
Sign in to run your code.

Output

Press Run to see what your program prints.