Topic 7.1
Why Models Need Tools
In one line
A model only produces text: it has no live data, it is unreliable at exact maths and it cannot act on the world. Tool calling lets the model ask for a function and lets your code run it, so the model supplies judgement and your code supplies facts and actions.
Think of it like this
A clever manager who sits in a room with no phone, no computer and no calculator. Ask for yesterday's sales and they will describe what sales usually look like. Hand them an assistant who can look things up, and they say 'please get yesterday's sales for Pune', read the number you bring back, and give a correct answer. The manager decides what to ask for. The assistant does the doing. A model with tools works the same way.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Tool (function calling)
- A function in your code that you describe to the model so it can ask you to run it.
- Tool call
- The model's request to run a tool: a name, the arguments and an id. It is only text until your code acts on it.
- Knowledge cutoff
- The date after which the model's training data has nothing. It cannot know later events by itself.
- Hallucination
- A fluent, confident answer that is false, because the model predicts likely text and does not look facts up.
- Agent
- A program that lets a model take several tool-using steps toward a goal.
- Registry
- Your table of allowed tool names and the functions behind them. Only what is in it can run.
Step by step
01Step 1: a model on its own
The first example asks a scripted fake model for a product and for the time. Its answers are fluent and wrong, or stale. A real model behaves the same way: nothing in the model can fetch the truth.
02Step 2: let code do what code does well
multiply(48271, 91) gives 4392661 in one line, every time. The model's 4392000 is close enough to look right. This is why exact work belongs in code.
03Step 3: the model asks
Instead of answering, the model replies with an assistant message whose content is null and whose tool_calls list holds one call: an id, a function name and arguments. Look closely at arguments: it is a string that contains JSON, not an object. You must parse it.
{
"role": "assistant",
"content": null,
"tool_calls": [
{ "id": "call_1", "type": "function",
"function": { "name": "multiply", "arguments": "{\"a\":48271,\"b\":91}" } }
]
}04Step 4: your code acts
Look the name up in a registry, parse the arguments, run the function. Here the registry has one entry. The next topics add validation, errors and limits.
05Step 5: send the result back
The result goes back as a message with role: "tool" and the same tool_call_id, so the model knows which request it answers. The model reads it and writes the final answer (Topic 7.2 runs this to the end).
Try it yourself
- 1
Predict, then run
Before you run the second example, write down what the
sent back:line will contain. Then run it in Python, JavaScript and Java and compare the three outputs. - 2
Add a second tool
Add a function
add(args)to the registry and change the scripted reply to ask foraddwith{"a":2,"b":3}. Run it. Nothing else in the code should need to change: that is the point of a registry. - 3
Ask for a tool that does not exist
Change the tool name in the scripted reply to
delete_everything. What happens at the registry lookup? Topic 7.4 turns this into a proper refusal.
Code & diagrams
The fake model returns scripted answers, like a real model that has no tool: fluent text, no way to check it.
Expected output
model alone: 48271 * 91 = 4392000
real answer: 4392661
model is right: no
time question: It is 14:05 in Pune.
(the time was written in advance: no clock is connected)The reply is a scripted assistant message in the real OpenAI-compatible shape. In Java the course's Json helper stands in for Jackson.
Expected output
model sent: {"role":"assistant","content":null,"tool_calls":[{"id":"call_1","type":"function","function":{"name":"multiply","arguments":"{\"a\":48271,\"b\":91}"}}]}
my code runs: multiply with a = 48271 and b = 91
sent back: {"role":"tool","tool_call_id":"call_1","content":"4392661"}Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Trust the model's arguments
In the second example, change the scripted arguments to '{"a":"ab","b":3}' (the model sent a text where a number is expected). Run the Python version.
Myth vs fact
Myth
The model runs the function.
Fact
The model only writes a request. Your program decides whether and how to run it, which is why you can validate, refuse and log.
Myth
Bigger models never need tools.
Fact
Size does not give a model live data or the ability to act, and even the best models are unreliable at long exact arithmetic. Tools fix a different problem from size.
Myth
The arguments are a JSON object.
Fact
In the OpenAI-compatible shape (the request and reply format many providers copy, Topic 2.5) they are a JSON string. It can be malformed or contain the wrong types, so parse it inside a try block and validate it.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Tool descriptions and schemas are part of the prompt and are billed as input tokens on every request. Ten verbose tools can cost more than the user's question. Send only the tools relevant to the current task.
- ▸
A tool call is a second network round trip to the model (one to get the call, one to get the answer). Budget latency for it: a two-step loop is roughly twice the model time plus your tool's time (Topic 6.3). Recall from Topic 6.1 that the second request also pays again for the whole first request as input tokens.
- ▸
Treat everything the model sends as user input, because a user can steer it (prompt injection, Phase 10). The tool runner is where you enforce what is allowed.
Remember this
- 1
Why this exists. A large language model (LLM) is a function from text to text. It was trained on data up to a knowledge cutoff (the date after which it has seen nothing), so it cannot know today's weather, your orders or the current price of anything. It predicts plausible text, so it can write '48271 * 91 = 4392000' with full confidence, and be wrong. And text cannot send an email or change a database row. Three gaps: no live data, no exact computation, no actions.
- 2
The idea. You describe some functions to the model in the request (a name, a plain-English description and a JSON Schema for the arguments: a standard way to describe the shape of JSON, taught in Topic 4.4). When the model decides it needs one, it does not answer. It replies with a tool call: the function name and the arguments. Your program runs the function itself and sends the result back as a new message. Then the model writes the final answer using the result. The model asks. Your code acts.
- 3
The model never runs anything. This is the most important sentence in the phase. A tool call is only text, a request from the model. If your program ignores it, nothing happens. If your program runs whatever it asks without checking, the model (or a user who tricked it) controls your program. So tool calling is mostly about what your code does between 'the model asked' and 'the function ran': find the tool in a registry, validate the arguments, limit the damage, report errors (Topics 7.3 to 7.7).
- 4
Tools are for facts and actions, not for thinking. Good tools: look up a record, search documents, call an API, do exact arithmetic, run a query, send a message with permission. Bad tools: anything that just rephrases what the model could say itself. Every tool you add costs tokens (its description is sent with every request, Topic 7.3) and gives the model one more way to go wrong, so keep the list short and clear.
- 5
Two failure modes without tools, both silent. Ask a model with no tool for the weather: it may invent a pleasant answer. Ask it for exact arithmetic: it may be close and wrong. Neither raises an error. The first example below shows a model that is confidently wrong twice, and your code that gets the right answer in one line. Recall from Topic 0.3: if plain code can compute it exactly, let code compute it, and let the model call that code.
- 6
Where this is going. The loop in the next topic (ask, run, send back, repeat) is the core of every agent: a program that lets a model take several steps toward a goal. The coming AI Agents course builds on it. Here you build it by hand, because the libraries that wrap it (the OpenAI and Anthropic SDK helpers, LangChain4j, the Vercel AI SDK) make sense only once you have seen what they hide.
Explain it without notes
Why can't a model on its own tell you today's weather or multiply two five-digit numbers reliably?
Who actually runs a tool in function calling, and why does that matter for security?
What does the arguments field of a tool call contain, and what should your code do with it?
Practice
Write parse_arguments(text) that returns the dict for a tool call's arguments string, or None when the text is not valid JSON or is not a JSON object.
Write make_tool_message(call, result) that returns the tool message dict for a tool call dict call (with an id) and a result string.
Trade-offs
- ↔
A tool gives exact, current results but adds a round trip, tokens for the description and one more thing that can fail.
- ↔
More tools make the model more capable and make wrong choices more likely; a short, clear list usually beats a long one.
- ↔
Letting code do exact work removes a class of errors but moves the responsibility for safety into your tool runner.
Done when you can
Done when you can name the three gaps (live data, exact computation, actions) that tools fill.
Done when you can say who runs a tool call and why that matters.
Done when you can point to the
tool_callsandargumentsfields and say what typeargumentsis.Done when you can decide whether a task needs a tool or just a prompt.