Direct answer vs step by step, then verify
The fake model is scripted to get the direct answer wrong and the step-by-step answer right. Real models vary, so always verify.
For multi-step problems, asking the model to work step by step often makes it more accurate. Reasoning models do this on their own. Show users the final answer, check exact values in code, and never treat the written reasoning as proof.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write final_answer(reply) that returns the integer after the last line starting with Answer:, or None if there is no such line.
Write check_answer(reply, expected) that returns "correct", "wrong" or "no answer".
Explain it without notes
Why does asking a model to work step by step often improve accuracy?
When should you avoid chain-of-thought prompting?
Why should you never show a model's raw reasoning to users as the real explanation?
How do you safely get a final number out of a step-by-step reply?
Expected output
direct: hidden steps=0, answer=12, correct=no
shown to the user: Answer: 12
step by step: hidden steps=3, answer=14, correct=yes
shown to the user: Answer: 14