Intermediate
Validate a JSON answer
Write validate(raw, n_sources) that returns a list of problem strings (empty means valid). raw is the model's reply as text.
In order: if raw is not valid JSON, return ["not valid JSON"]. If the value is not an object with a string answer and a list citations of whole numbers, return ["bad shape"]. Otherwise collect problems: citation out of range: 3,5 (the out-of-range numbers in order, joined by commas, when any are below 1 or above n_sources) and an answer without citations must be the refusal (when citations is empty and the answer is not exactly REFUSAL).
What Check tests
- A good reply has no problems
- Text that is not JSON
- Wrong shape
- Out-of-range citations are named
- No citations is allowed only for the refusal
Stuck? One hint at a time
The solution unlocks after your first Check. Try first: that is where the learning happens.
main.py
Sign in to run your code.
Output
Press Run to see what your program prints.