Unit-test the feature with a fake model
A scripted FakeModel returns replies in order and records requests. Five tests run in milliseconds with no key and no network.
Most tests of an LLM feature should not call a real model. A fake model with scripted replies tests your code (parsing, retries, fallbacks, prompt assembly), and recorded real responses replayed by a request hash test it against realistic data, for free and in milliseconds.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write ScriptedModel(replies) with a complete(text) method that returns the replies in order and raises IndexError with the message script exhausted when asked for more. Call it three times with two replies and print what happens.
Write make_key(system, user, temperature) that returns the tiny_hash of system + "|" + user + "|" + str(temperature) (hash: h = 7, then h = (h * 31 + ord(ch)) % 1000000007 for each character). Print the key for ("s", "u", 0) and show that changing the temperature to 1 changes it.
Explain it without notes
What bugs does a fake model catch, and what does it not catch?
Describe record and replay for model calls, and what goes into the key.
Why should a changed prompt make a replay fail?
Which kinds of test would you run on every commit, and which on a schedule?
Expected output
PASS a clean reply is returned
PASS chatter is retried once
PASS a category outside the list is rejected
PASS two bad replies fall back
PASS the request carries the prompt and temperature 0
5 passed, 0 failed