AskDocs version 8: gate, budget, labelled sources, checked citations
The scores in SCORES are real cosines from the search lab, rounded. The model is a scripted stand-in that follows the prompt's rules. Re-measure the threshold on the whole handbook before trusting 0.40.
AskDocs version 8 adds the whole generation side: a refusal gate on a measured threshold, a token-budgeted prompt with numbered sources and dates, a citation check, and a visible warning when a source is out of date.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Add a question "Is the handbook available in French?" to the example's SCORES with D1 at 0.12 and D3 at 0.10. Predict the output of answer, then run it.
Write explain_refusal(hits, threshold) returning the sentence best score X.XX is below Y.YY for a list of (doc_id, score) pairs, best first. Print it for [("D1", 0.27), ("D6", 0.24)] and 0.40.
Explain it without notes
Walk through the stages of AskDocs version 8 for the pass-price question.
Why is the scooter question refused before the model is called?
What does the verifier check, and what does it not?
Which parts of the answer path are real in the example and which are stand-ins?
Expected output
Q: I forgot my login, how do I get back in?
hits: D3 0.47, D1 0.11, D6 0.11
kept: D3 (39 of 120 tokens)
citations: ok
A: To reset your password, open the app, tap Forgot password and follow the link we email you. [1]
Q: What does the monthly pass cost?
hits: D16 0.76, D8 0.75, D6 0.22
kept: D16, D8 (58 of 120 tokens)
citations: ok
A: The monthly pass costs 25 dollars and includes unlimited rides of up to 45 minutes. [2] ([1] is an older version and no longer applies.)
Q: Can I get my money back if I cancel?
hits: D1 0.53, D6 0.33, D3 0.23
kept: D1 (39 of 120 tokens)
citations: ok
A: A ride cancelled at least 2 hours before its start time is refunded in full. [1]
Q: Do you rent scooters?
hits: D1 0.27, D6 0.24, D16 0.23
gate: best score 0.27 is below 0.40, model not called
A: I could not find that in the handbook.