AskDocs as a service: caches, budget, trace and health check
Retrieval is the keyword toy used since Phase 0 and the times are simulated. Replace the search and the generate step and the surrounding service code stays the same.
AskDocs becomes a service: an ingestion path and a query path with an answer cache that is invalidated by document, a time budget that skips re-ranking when search is slow, a trace line for every request and a health check with a canary question.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write stages_run(search_ms, budget_ms, rerank_ms=80, generate_ms=300) returning the list of stages that run (search always, rerank only if search plus rerank plus generate fit the budget, generate always). Print it for search times 20 and 150 with a budget of 500.
Write hit_rate(hits, requests) returning a percentage as a float, and 0.0 when there were no requests. Print it with one decimal for (1, 5) and (0, 0).
Explain it without notes
What does AskDocs-as-a-service add around retrieval, and why at the edges?
How does the example avoid serving a stale price after a document edit?
What does the time budget do, and what does it never skip?
What should a health check test for a RAG service?
Expected output
health: FAIL: the index is empty
health: ok (index version 3, canary found D8)
trace: cache=miss search=20 rerank=80 generate=300 total=400 | The monthly pass costs 25 dollars and includes unlimited rides of up to 45 minutes. [D8]
trace: cache=hit total=1 | The monthly pass costs 25 dollars and includes unlimited rides of up to 45 minutes. [D8]
edit D8: dropped 1 cached answer(s)
trace: cache=miss search=20 rerank=80 generate=300 total=400 | The monthly pass costs 30 dollars and includes unlimited rides of up to 45 minutes. [D8]
trace: cache=miss search=150 rerank=skipped generate=300 total=450 | Shops open at 7 am and close at 9 pm every day [D2]
trace: cache=miss search=20 total=20 | I can't find that in the handbook.
requests=5 cache_hits=1 skipped_rerank=1