LLM Engineering · Exercise 6 of 6 Previous
Advanced
Answer with guards
Write answer(question, retrieve, model, min_score=0.3). retrieve(question) returns a list of (score, text) pairs, best first. Keep only the texts whose score is at least min_score.
If none are left, return REFUSAL without calling the model. Otherwise build the prompt with the given build_prompt(question, texts) and call model(prompt). If check_reply(reply, len(texts)) is neither "ok" nor "refusal", return REFUSAL; otherwise return the reply.
What Check tests
- A good reply is returned unchanged
- The model sees numbered sources and the question
- No evidence: refuse and never call the model
- Weak hits are dropped, strong ones kept
- A reply without a citation is replaced by the refusal
- A citation to a missing source is replaced by the refusal
- A refusal from the model is passed through
Stuck? One hint at a time
The solution unlocks after your first Check. Try first: that is where the learning happens.
main.py
Sign in to run your code.
Output
Press Run to see what your program prints.