Command Palette

Search for a command to run...

Hectal
PHASE 2Beginner ~19 min· topic 3 of 9

Topic 2.3

Reading the Response

In one line

A reply is more than text: finish_reason says why the model stopped (finished, cut off, wants a tool, blocked) and usage says how many tokens you paid for. Read every field defensively, because any of them can be missing, null or empty.

0/9 · 0%

Think of it like this

A parcel delivery slip. The parcel is the answer, but the slip tells you more: delivered, left with a neighbour, damaged, or returned because nobody was home. A careful person checks the slip before opening the box. A model reply carries a slip too, and ignoring it is how programs treat half an answer as a whole one.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

choices
The list of possible replies in a response. With the usual request it has one item.
finish_reason
Why the model stopped: stop (done), length (ran out of tokens), tool_calls (wants a tool) or content_filter (blocked).
usage
The token counts for the call: input (prompt_tokens), output (completion_tokens) and their sum.
max_tokens
The most tokens the model may write in one reply. Hitting it gives finish_reason: length.
Reasoning tokens
Hidden tokens some models spend thinking before the answer. They are billed as output and count toward the limit.
Tool call
A reply that asks your program to run a named function with arguments, instead of giving text.
Defensive parsing
Reading a reply so that a missing, null or odd field gives a clear result instead of a crash.

Step by step

01Step 1: look at the shape

A normal reply holds one choice with the answer and finish_reason: "stop", and a usage object. The first scripted reply in the example is exactly that, printed as one line: OK, the text, and the token total.

Step 1: look at the shapediagram
Rendering diagram…

02Step 2: the cut-off reply

The second reply ends in the middle of a sentence and has finish_reason: "length". The text looks plausible, which is why you must check the reason. The fix is a bigger max_tokens, a shorter answer request, or asking the model to continue.

03Step 3: no text at all

A tool call has content: null and a tool_calls list. A filtered reply has empty content. Code that does content.strip() on null crashes with an error unrelated to the real situation.

04Step 4: missing pieces

Some servers leave out usage; the reader prints tokens=unknown instead of failing. An empty choices list is reported as an error. Each of these has been seen in real traffic.

05Step 5: look at a real reply

Open the lab below, press Send and open Raw HTTP. Find choices[0].finish_reason, usage.completion_tokens and, for the reasoning model, how many of those tokens were hidden reasoning. Then lower max tokens and send again to provoke length.

06Step 6: turn it into rules

Write down what your feature does for each case: stop with text (use it), stop with empty text (retry once, then fallback), length (raise the limit or continue), tool_calls (run tools), content_filter (safe message), anything else (log and fail clearly).

Try it yourself

  1. 1

    Predict each line

    Before running the example, predict the verdict and token total for all six replies. Pay attention to the two that have no text and the one without usage.

  2. 2

    Add a seventh reply

    Add a reply whose finish_reason is "function_call" (an old name that some servers still send). What does the reader print? Decide whether to treat it as a tool call.

  3. 3

    Provoke length

    In the lab, set max tokens to 40 and send. Is content empty or cut off? Open the usage details: how many tokens were reasoning? Change reasoning effort to low and try again.

Code & diagrams

Reading six kinds of reply without crashing New tab

The replies are written out as raw JSON text, the way they arrive. Java reads them with the course's Json helper; real Java projects use Jackson.

Sign in to run this example in your browser.

Expected output

normal     -> OK: Rain all week. | tokens=25
cut off    -> CUT OFF (raise max_tokens): Rain is expected, but the | tokens=26
tool call  -> TOOL CALL: get_weather | tokens=52
filtered   -> BLOCKED by the provider's filter | tokens=20
no usage   -> OK: Rain all week. | tokens=unknown
no choices -> ERROR no choices in the reply
Read a real replyreal model

Try: Press Send, then open Raw HTTP and find finish_reason and usage. Then lower max tokens to 40 and send again: what is finish_reason now, and how many completion tokens were used?

user

Goals

  • The reply finished on its own (finish_reason is stop)
  • The answer is not empty

Sign in to send this request to a real model (free).

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Trust the text

In the Python reader, delete the finish_reason checks: return "OK: " + content for every reply that has choices (keep the tokens part).

terminal
$ Run
── what you'll see ──
normal -> OK: Rain all week. | tokens=25
cut off -> OK: Rain is expected, but the | tokens=26
tool call -> OK: | tokens=52
filtered -> OK: | tokens=20
no usage -> OK: Rain all week. | tokens=unknown
no choices -> ERROR no choices in the reply

Myth vs fact

Myth

If the call returned 200, the text is the full answer.

Fact

200 only means the request worked. finish_reason: length means the text was cut off, and a filter can blank it.

Myth

content is always a string.

Fact

It can be null (tool calls) or empty. Treat it as optional.

Myth

usage is always there and exact for my estimate.

Fact

Some servers omit it, and real counts differ from character-based estimates, especially with reasoning tokens.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

  • ▸

    Chart finish_reason counts per feature. A rising share of length means prompts or outputs have grown; a rising content_filter share points to an input or policy change.

  • ▸

    Log usage with the model name for every call. Cost per feature is the sum of prompt_tokens times the input price plus completion_tokens times the output price (Topic 6.1).

  • ▸

    n > 1 returns several choices; most providers do not support it for every model. Code that assumes choices[0] is fine for n = 1; if you ever request more, choose by index, not by position luck.

Remember this

  1. 1

    Why this exists. The text alone cannot tell your program whether it is complete. A reply that stops mid-sentence looks like any other string. So the API reports the reason it stopped in finish_reason, and the number of tokens used in usage. Your program must read both on every call, because they decide what to do next: use the text, ask for more, run a tool, or show an error.

  2. 2

    The shape. The top level has choices (a list; with the normal request there is one item) and usage. Each choice holds a message (role, content, and sometimes tool_calls) and a finish_reason. The text you want is choices[0].message.content. Other fields (id, model, created) help with logging.

  3. 3

    **finish_reason values.** stop: the model finished on its own, or hit a stop sequence (a piece of text you told it to stop at); the normal case. length: it ran out of the max_tokens you allowed (or the context window) mid-answer; the text is incomplete. tool_calls: it did not answer but asks your program to run a tool (Phase 7); content is often null. content_filter: the provider's safety filter blocked or cut the reply. Unknown values can appear when providers add new ones, so always keep an else branch.

  4. 4

    **usage and money.** prompt_tokens is what you sent (the whole message list), completion_tokens is what the model wrote, total_tokens is the sum. Providers bill input and output tokens at different prices, so keep both numbers (Topic 6.1). Reasoning models also spend hidden reasoning tokens before the visible answer; they count as output and can use the whole max_tokens, leaving an empty content with finish_reason: "length" (Topic 1.8). The lab below shows this with a real model.

  5. 5

    Read defensively. In real traffic content can be null (tool calls) or "" (filtered or exhausted), usage can be absent (some compatible servers omit it, and streaming replies only include it if asked), and choices can be empty. The example reads six replies with one function that never crashes and says in plain words what happened. Reading reply["choices"][0]["message"]["content"] directly is fine in a demo and a crash in production.

  6. 6

    **Decide on finish_reason, not on text.** Treat length as a failure or a prompt to continue; never show it to users as if it were complete. Treat an empty content with stop as a failure too, and log it. Counters for each finish_reason are one of the cheapest and most useful production metrics (Topic 11.4).

Explain it without notes

01

What does finish_reason: "length" mean and how should code react?

02

Which fields of a chat reply can be missing or null, and why does it matter?

03

How is the bill for one call computed from the reply?

Practice

01

Write is_complete(reply_json_text) that returns True only when the first choice has finish_reason equal to stop and non-empty content.

02

Write total_tokens(reply_json_text) that returns usage.total_tokens, or prompt_tokens + completion_tokens when total_tokens is missing, or None without usage.

Trade-offs

  • ↔

    Strict handling of every case adds code; it replaces silent wrong answers with clear failures and metrics.

  • ↔

    Auto-continuing a cut-off reply recovers the text but costs a second call and can drift; raising max_tokens is simpler when the limit was too tight.

Done when you can

  • Done when you can name the four finish_reason values and the action for each.

  • Done when you read usage and can compute the cost of a call.

  • Done when your reader survives null content, missing usage and empty choices.

  • Done when you can find these fields in the lab's raw response.