PHASE 7Intermediate ~22 min· topic 2 of 9

Topic 7.2

The Tool-Call Loop by Hand

In one line

The tool loop is: send the conversation and the tools, run any tool calls the model returns, append the results as tool messages, and ask again until the model answers in plain text. A step limit stops a model that never finishes.

Think of it like this

Ordering at a counter where the cook can't leave the kitchen. You say what you want. The cook shouts 'ask the shop next door if they have saffron'. You run next door, come back with 'yes, a small jar', and the cook cooks. Sometimes the cook shouts a second errand. The conversation goes back and forth until the cook says 'here is your meal'. You also decide that after ten errands you walk out.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Tool loop: Repeat: ask the model, run the tool calls it returns, send the results back, until it answers in plain text.
tool message: A message with role: tool and a tool_call_id that carries one tool's result back to the model.
tool_call_id: The id the model gave a tool call. The tool message must repeat it so the model can match result to request.
Step limit: The maximum number of loop rounds. It stops a model that never finishes.
tool_choice: A request field that says whether the model may, must or must not call tools.
finish_reason: Why the model stopped. tool_calls means it wants a tool run. stop means it finished.

Step by step

01Step 1: the first request

The request holds the user's message and the tools list. The model sees the descriptions and decides: answer directly, or ask for a tool.

02Step 2: read the reply

The fake model replies with tool_calls. A real model also gives finish_reason: "tool_calls". The code checks reply.get("tool_calls"): empty means a final answer.

03Step 3: keep the assistant message, run the call

Append the reply to messages unchanged. Parse the arguments, look the function up in the registry, run it, and turn the result into a JSON string.

04Step 4: add the tool message, ask again

Append {"role":"tool","tool_call_id":"call_1","content":"…"} and send the whole list again. The model now sees the question, its own request and the result, and writes the answer.

Step 4: add the tool message, ask againdiagram

Rendering diagram…

05Step 5: add the safety net

The second example wraps the cycle in for step in range(1, max_steps + 1). A model that finishes returns early. A model that keeps asking is stopped at the limit and the function returns None so the caller can handle it.

06Step 6: a real model

In the lab, press Send. A real model sees the get_weather tool and (usually) replies with a tool call instead of text. Use Answer the tool call, edit the fake result, and send again to get the final answer.

Try it yourself

1
Count the messages
Run the first example and read the last line. Predict what it would print if the user asked a question that needed no tool, then change the script to a single text reply and check.
2
Lower the step limit
In the second example, change the limit in the first call from 5 to 1. The model that would have finished at step 2 is now stopped. What does your caller get back, and what should a real product show the user?
3
Use the lab
In the lab, change the question to 'Say hello' and send. Does the model still call the tool? Then add 'Always use the tool' to the system prompt and compare. A prompt only asks the model; the request field tool_choice: "required" is what forces a call.

Code & diagrams

One full round trip New tab

The scripted fake model stands in for the API. Each step prints what a real log of this loop would show.

Expected output

step 1: model asks get_weather {"city":"Pune"} (id call_1)
step 1: tool result {"city":"Pune","temp_c":31,"sky":"sunny"}
step 2: model answers: It is 31 C and sunny in Pune.
roles in the conversation: user, assistant, tool, assistant

A loop with a step limit New tab

The same loop as a function. The second fake model asks for the same tool forever.

Expected output

a model that finishes:
  finished at step 2
  answer: 31 C in Pune.
a model that never stops asking:
  stopped: step limit 3 reached
  answer: (none)

A real tool callreal model

Try: Press Send. The model should answer with a tool call (see the Tool calls box and finish_reason: tool_calls) instead of text. Then press 'Answer the tool call', replace the placeholder result with {"temp_c": 31, "sky": "sunny"} and send again: the model should now answer in words.

System

user

Modelsmall, fast, reasons firstTemperature · 0.0Max tokens (1–1024)Reasoning effort Stream (SSE) JSON mode Tools: get_weather

Goals

The model calls get_weather
finish_reason is tool_calls

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Forget to append the assistant message

In the second example's run_loop, delete the line messages.append(reply) and add print(len(messages)) just before the line that prints finished at step. (In a correct loop that count is 4 at that point.)

terminal

$ Run

── what you'll see ──

a model that finishes:

finished at step 2

answer: 31 C in Pune.

a model that never stops asking:

stopped: step limit 3 reached

answer: (none)

Myth vs fact

Myth

The model remembers the tool result from last time.

Fact

Every request starts from nothing. If you do not resend the earlier messages, the model has no idea a tool was ever called.

Myth

A step limit is only for buggy models.

Fact

It is also your cost cap and your defence against steering by hostile content. Every loop that calls a model needs one.

Myth

The model returns either text or a tool call, never both.

Fact

A reply can contain text and tool calls together, and some models return several calls. Check tool_calls first, and keep any text.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Cost grows faster than steps: each step resends the whole growing message list, so a ten-step loop pays for the early messages ten times (the input tokens grow roughly with the square of the step count when every step adds a similar amount). Use prompt caching where the provider offers it (Topic 6.6) and trim old tool results.
▸
Log every step with its number, tool name, argument size, duration and outcome. When a loop misbehaves, the trace of steps is the first thing you read (Phase 11).
▸
Decide the failure policy before launch: when the limit is hit, return a partial answer with an apology, or an error the caller can retry. Never let the loop run silently or raise an unhandled error to the user.

Remember this

1
Why a loop. One question may need one tool call, several in a row (find the order, then look up its shipment) or none. Your code cannot know in advance, so it repeats the same small cycle until the model stops asking. This cycle is the basis of every agent framework.
2
The cycle. (1) Send messages plus tools to the model. (2) Look at the reply. If it has no tool_calls, its content is the final answer: stop. (3) Otherwise append the assistant message to messages exactly as received, run each call, append one role: "tool" message per call with the matching tool_call_id, and go to step 1. The model sees its own request followed by the results, which is how it knows what happened.
3
The request shape. Tools go in a tools list: {"type":"function","function":{"name":"get_weather","description":"…","parameters":{JSON Schema}}}. You can steer with tool_choice: "auto" (the model decides, the default), "none" (no tools), "required" (it must call at least one), or one named function written as {"type":"function","function":{"name":"get_weather"}}. In the reply, finish_reason is tool_calls when the model stops to ask for a tool, and stop when it has finished (Topic 2.3). The loop in this topic acts on whether tool_calls is present and non-empty, because that field is what you must answer.
4
Why the assistant message must be kept. The model has no memory between calls (Topic 1.5). If you drop its tool-call message and send only the tool result, the result refers to a request the model cannot see, and APIs may reject the message list. Always append the assistant message, then its tool messages, in that order.
5
The step limit. A model can call tools forever: it may keep retrying a failing call, bounce between two tools, or be steered by a malicious document. Each step costs money and time. Always set a maximum number of steps (5 to 10 is typical for simple assistants) and decide what to do when it is reached: return what you have with a clear message, never loop silently. The second example shows a model that never stops and the limit catching it.
6
Everything else goes in the same loop. Validation, errors fed back, several calls per turn, confirmation, logging, timeouts and budgets are all additions inside this cycle. Keep the loop small and readable: it is the part you will debug most. The lab below lets a real model produce a real tool call, then lets you play the tool and send the answer back.

Explain it without notes

Describe the tool-call loop step by step.

Why must the assistant message be appended before the tool message?

Why does every tool loop need a step limit, and what should happen when it is reached?

Practice

Write run_loop(script, registry, max_steps) that returns the final text of the first scripted message without tool_calls, running each call with registry[name](args), and returns None if max_steps is reached. Messages in script use the real shape.

Write count_steps(messages) that returns how many assistant messages a message list contains.

Trade-offs

↔
A higher step limit allows harder tasks and raises worst-case cost and latency; a low one is safe and fails more often.
↔
Resending the full history gives the model complete context and makes each step more expensive; trimming saves money and can drop something needed.
↔
Writing the loop by hand gives full control and more code; SDK helpers are shorter and hide what happens when something goes wrong.

Done when you can

Done when you can write the loop from memory: send, check tool_calls, append, run, append, repeat.
Done when your loop always appends the assistant message before its tool messages.
Done when every tool message carries the right tool_call_id.
Done when your loop has a step limit and a planned behaviour when it is hit.
Done when you have seen a real model produce a tool call in the lab.

Command Palette

Words you'll meet

Step by step

01Step 1: the first request

02Step 2: read the reply

03Step 3: keep the assistant message, run the call

04Step 4: add the tool message, ask again

05Step 5: add the safety net

06Step 6: a real model

Try it yourself

Count the messages

Lower the step limit

Use the lab

Code & diagrams

Break it on purpose

Forget to append the assistant message

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can