Command Palette

Search for a command to run...

Hectal
Phase 0: Start Here: What RAG Is and What You Will Build
RAG and Vector Search · Exercise 4 of 5 PreviousNext
Intermediate

Check an answer's citations

Write cited_ids(answer) returning every id written in square brackets, in order ("See [D8] and [D6]." gives ["D8", "D6"]). Ignore an unclosed [.

Write review(answer, retrieved) returning "ok" when answer equals REFUSAL; otherwise "no evidence" when retrieved is empty, "no citation" when nothing is cited, "unknown source" when any cited id is not in retrieved, else "ok". Check in that order.

What Check tests

  1. cited_ids finds ids in order
  2. An unclosed bracket is ignored
  3. A made-up citation is caught
  4. An answer without a citation is caught
  5. Answering with nothing retrieved is caught; refusing is fine
  6. A correct answer passes

Stuck? One hint at a time

The solution unlocks after your first Check. Try first: that is where the learning happens.

main.py
Sign in to run your code.

Output

Press Run to see what your program prints.