Advanced
Name the stage that broke
Write triage(expected, indexed, ranking, k, sent, answer, fact) that returns the first broken stage as one of these strings: not indexed, not retrieved, ranked too low, cut by budget, ignored by the model, or ok.
indexed is a set of ids in the index. ranking is the list the search returned. k is how many are eligible to be sent. sent is the list of ids that actually went into the prompt. fact is text the answer must contain.
Check in pipeline order: not in indexed; not in ranking; its place in ranking greater than k; not in sent; fact not in answer.
What Check tests
- A working run is ok
- Missing from the index comes first
- Indexed but not returned
- Returned below the cutoff
- Inside k but not sent
- Sent but the answer lacks the fact
Stuck? One hint at a time
The solution unlocks after your first Check. Try first: that is where the learning happens.
main.py
Sign in to run your code.
Output
Press Run to see what your program prints.