PHASE 1Beginner ~19 min· topic 8 of 10

Topic 1.8

Reasoning Models

In one line

Reasoning models write hidden thinking tokens before the visible answer. That helps on multi-step problems, and it costs time and money, and the thinking shares your max_tokens budget, which creates a classic trap: an empty reply with finish_reason: length.

Think of it like this

A maths exam. One student writes the answer straight away. Another uses the margin for working: writes the steps, checks them, then copies the final answer. For hard questions the second student is right more often, but needs more time and more paper. A reasoning model writes its working before it answers.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Reasoning model: A model trained to write hidden step-by-step thinking before it answers.
Reasoning tokens: The hidden thinking tokens. They count as output tokens for price and for max_tokens.
Chain of thought: Intermediate steps written out before the final answer.
Reasoning effort: A setting that tells the model how much to think: low, medium or high.
finish_reason: Why generation stopped: stop (finished), length (hit max_tokens), tool_calls, content_filter.
Time to first token: How long you wait before the first visible token. It grows with thinking time.

Step by step

01Step 1: a puzzle that needs steps

"Anna is twice as old as Ben. In six years she will be one and a half times his age. How old is Ben?" To answer you must set up an equation and solve it. A quick guess is often wrong.

02Step 2: thinking first, answer second

A reasoning model writes its working, perhaps hundreds of tokens, and then the short visible answer. The working is billed as output and counts toward max_tokens.

Step 2: thinking first, answer seconddiagram

Rendering diagram…

03Step 3: simulate it

The example fakes a model that needs 40, 120 or 300 thinking tokens by effort and then a 25-token answer. It reports finish, the hidden and visible counts, and a cost at an example price. Read the five rows in order.

04Step 4: read the trap

high with max_tokens=100 spends all 100 tokens thinking: finish is length, the answer is empty, and you still pay for 100 tokens. high with 320 finishes thinking and then runs out in the middle of the answer. 400 is enough (the true minimum is 300 + 25 = 325).

05Step 5: run it for real

The lab below sends the puzzle to a real reasoning model. Send it with effort low, read the reasoning tokens in the usage, then send again with effort high. Then set a small max_tokens such as 100 and see the empty reply for yourself.

Try it yourself

1
Find the minimum budget
For effort high, find the smallest max_tokens that gives finish=stop in the first example. It should be 325. Check by adding a row.
2
Cost of thinking
Compare the cost of low with max_tokens 400 and high with 400. How much of the high-effort bill is hidden thinking?
3
Use the lab
Run the lab with low effort, then high. Write down total tokens, time and the answer for each. Was the extra thinking worth it for this puzzle?

Code & diagrams

Hidden thinking uses the same token budget New tab

The thinking sizes are made up. A real model decides by itself how long to think, within the effort you set.

Expected output

effort=low    max_tokens=100 -> finish=stop   reasoning= 40 answer=25 (complete) cost=0.000130
effort=high   max_tokens=100 -> finish=length reasoning=100 answer= 0 (empty) cost=0.000200
effort=medium max_tokens=200 -> finish=stop   reasoning=120 answer=25 (complete) cost=0.000290
effort=high   max_tokens=320 -> finish=length reasoning=300 answer=20 (cut off) cost=0.000640
effort=high   max_tokens=400 -> finish=stop   reasoning=300 answer=25 (complete) cost=0.000650

Reasoning effort on a multi-step puzzlereal model

Try: Send it with reasoning effort low and read reasoning_tokens in the token usage. Then change the effort to high and send again: how many more tokens did it use and did the answer change? Finally set max tokens to 100 and send once more.

System

user

Modelsmall, fast, reasons firstTemperature · 1.0Max tokens (1–1024)Reasoning effort Stream (SSE) JSON mode

Goals

The reply finished on its own (finish_reason is stop)
The answer says Ben is 6

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Bill only the visible tokens

In the Python version change (reasoning + answer) in the cost line to just answer.

terminal

$ Run

── what you'll see ──

effort=low max_tokens=100 -> finish=stop reasoning= 40 answer=25 (complete) cost=0.000050

effort=high max_tokens=100 -> finish=length reasoning=100 answer= 0 (empty) cost=0.000000

effort=medium max_tokens=200 -> finish=stop reasoning=120 answer=25 (complete) cost=0.000050

effort=high max_tokens=320 -> finish=length reasoning=300 answer=20 (cut off) cost=0.000040

effort=high max_tokens=400 -> finish=stop reasoning=300 answer=25 (complete) cost=0.000050

Myth vs fact

Myth

A reasoning model is just a bigger model.

Fact

It is trained to think before answering, and uses more tokens per answer. Size and reasoning are separate things.

Myth

If max_tokens is reached, I still get a partial answer.

Fact

With a reasoning model the budget may be spent on hidden thinking, and the content comes back empty.

Myth

Reasoning is always better.

Fact

On simple tasks it only adds delay and cost. Use it where steps matter, and verify results that matter.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
The name of the cap differs. OpenAI's reasoning models reject max_tokens and take max_completion_tokens, and Groq marks max_tokens as deprecated in favour of max_completion_tokens. Providers with an explicit thinking budget (a token count for the thinking) still keep it inside the overall output cap. Read the parameter table of your provider before you set it.
▸
Alert on finish_reason == "length" and on empty content. These are the two signals that a token budget is too small, and they look like successful calls in HTTP-level monitoring.
▸
Latency has two parts: thinking time (before any visible token) and generation time. Streaming does not hide thinking time. Time to first visible token can be many seconds at high effort, so design the UI for it (Phase 5).
▸
Route by difficulty: a cheap classifier or rule decides between a fast model and a reasoning model. Many products default to low effort and escalate only after a failed validation.
▸
Do not rely on reading the chain of thought. Providers may hide or summarise it, and it is not guaranteed to reflect the real computation. Treat it as a debugging aid at most.

Remember this

1
Why this exists. A normal model must produce each token after a single pass through the network, with a fixed amount of computing per token. A multi-step problem (a puzzle, a plan, a bug hunt) needs more thinking than one pass allows. If the model first writes out intermediate steps, those steps become part of the context and the next tokens can build on them. More generated tokens means more computation spent on the problem.
2
What a reasoning model does. It is trained (with reinforcement learning, which means rewarding the model when its final answer is right, on problems with checkable answers such as maths and code) to write a chain of thought first and then the final answer. Those thinking tokens are called reasoning tokens. Many APIs do not return them (or return only a summary), but they exist, take time before the first visible word, and you pay for them as output tokens. The usage of the reply shows how many there were.
3
Reasoning effort. Many reasoning models accept a setting such as reasoning_effort (low, medium, high). Low spends fewer thinking tokens, answers sooner and costs less. High thinks longer and is more reliable on hard problems. The setting names and values differ by provider, so check the docs for your model. The example shows the pattern with made-up numbers: 40, 120 and 300 thinking tokens.
4
The max_tokens trap. max_tokens limits all generated tokens, hidden ones included. If the model spends the whole budget thinking, there are none left for the answer: the reply has empty content and finish_reason: "length". Your code sees a successful call with nothing in it. Rows 2, 4 and 5 of the example's output show an empty answer, a cut-off answer and a complete one. Fix it by leaving a generous budget, lowering the effort, and checking finish_reason on every reply.
5
When to use one, and when not. Use reasoning for multi-step logic, maths, planning, tricky code and ambiguous tasks where mistakes are costly. Do not use it for simple extraction, classification, short rewrites or chat where speed matters: it is slower and more expensive for no gain. A good design routes easy requests to a fast model and hard ones to a reasoning model (Topic 12.6).
6
Thinking is not proof. Reasoning models still make mistakes, and the visible explanation may not match the real process. Verify results that matter with code (Topic 0.3) or a check, as you would a student's working.

Explain it without notes

Why do reasoning models help on multi-step problems?

What is the max_tokens trap and how do you avoid it?

When would you not use a reasoning model?

How do reasoning tokens show up in cost and latency?

Practice

Write fake_call(needed, answer_tokens, max_tokens) that returns (reasoning_used, answer_used, finish) using the rules in the example. Print the result for (300, 25, 100) and (300, 25, 400).

Write min_budget(needed, answer_tokens) and print it for needed=120 and answer_tokens=25. Add a 10 percent safety margin, rounded up, and print that too.

Trade-offs

↔
More reasoning gives better answers on hard problems and costs more tokens and time.
↔
A large max_tokens avoids the trap and allows a larger bill, so pair it with a budget guard.
↔
Routing easy and hard requests to different models saves money and adds a decision to build and test.

Done when you can

Done when you can explain what reasoning tokens are and how they are billed.
Done when you can reproduce and fix the max_tokens trap.
Done when you check finish_reason and empty content on every reply.
Done when you can decide whether a task deserves a reasoning model.

Command Palette

Words you'll meet

Step by step

01Step 1: a puzzle that needs steps

02Step 2: thinking first, answer second

03Step 3: simulate it

04Step 4: read the trap

05Step 5: run it for real

Try it yourself

Find the minimum budget

Cost of thinking

Use the lab

Code & diagrams

Break it on purpose

Bill only the visible tokens

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can