PHASE 7Intermediate ~22 min· topic 6 of 9

Topic 7.6

Several Tool Calls at Once

In one line

A model can return several tool calls in one reply. Each has its own id, they are independent of one another, so run them together to save time, and send back one tool message per call matched by id.

Think of it like this

A head chef who shouts three orders at once: 'soup to station one, salad to station two, bread to station three'. Three cooks work at the same time, and the meal waits for the slowest. Each plate carries a ticket number so it goes to the right table, whatever order the cooks finish in. The ticket number is the tool call id.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Parallel tool calls: Several tool calls in one assistant message, to be run at the same time.
tool_call_id: The id that ties a tool message back to the call it answers.
Concurrency: Several tasks in progress at once, finishing in any order.
Independent calls: Calls where none needs another's result, so they can run together.
Timeout: The longest time you will wait for one tool before giving up on it.
Idempotent: Safe to run twice: the second run changes nothing. Useful for writes that might be repeated.

Step by step

01Step 1: one reply, three calls

The assistant message holds three entries in tool_calls: two get_weather and one get_stock, with ids call_1 to call_3. A loop over tool_calls handles one call or many with the same code.

02Step 2: run them

Each call has its own simulated time. Run one after another the total is 570 ms. Started together, the whole batch takes as long as the slowest, 300 ms.

Step 2: run themdiagram

Rendering diagram…

03Step 3: results arrive in any order

In the second example the fastest call finishes first: call_2, call_3, then call_1. If you store results in arrival order and pair them with calls by position, every pair is wrong.

04Step 4: pair by id

Build a dictionary from tool_call_id to content, then walk the original calls in order and emit one tool message each. The result is right whatever the finish order.

05Step 5: the real thing

With Python asyncio.gather, JavaScript Promise.all or a Java ExecutorService, the library gives results back in the order of the calls. The shown code adds a timeout per call and turns any failure into that call's error message.

Try it yourself

1
Change the times
In the first example make the stock call take 50 ms. Predict both totals, then run. Which call decides the concurrent time now?
2
Break the pairing
In the second example change the times so the order of finishing is call_3, call_1, call_2. Which pairs does the wrong method get right by luck?
3
Count the messages
In the lab, how many tool calls did the model return and how many tool messages does 'Answer the tool call' create? They must match one to one.

Code & diagrams

Three tool calls in one turn New tab

Tool times are simulated with numbers, so the output is the same everywhere. The Java example is single-threaded, as all runnable Java in this course is.

Expected output

the model asked for 3 tools in one turn
call_1 get_weather {"city":"Pune"} -> {"city":"Pune","temp_c":31} (120 ms)
call_2 get_weather {"city":"Delhi"} -> {"city":"Delhi","temp_c":36} (150 ms)
call_3 get_stock {"symbol":"ACME"} -> {"symbol":"ACME","price":42} (300 ms)
one after another: 570 ms
all at once: 300 ms (the slowest call)
tool messages for: call_1, call_2, call_3

Pair results by id, not by position New tab

The finish order is simulated by sorting on the call times.

Expected output

finished in this order: call_2, call_3, call_1
pairing by arrival position (wrong):
  call_1 asked about Pune, got {"city":"Delhi","temp_c":36}
  call_2 asked about Delhi, got {"city":"Mumbai","temp_c":33}
  call_3 asked about Mumbai, got {"city":"Pune","temp_c":31}
pairing by id (right):
  call_1 asked about Pune, got {"city":"Pune","temp_c":31}
  call_2 asked about Delhi, got {"city":"Delhi","temp_c":36}
  call_3 asked about Mumbai, got {"city":"Mumbai","temp_c":33}

Running tool calls concurrently for realpython

Not run here: it needs real concurrency. Each version keeps the order of the calls, applies a deadline and turns a failure into an error message for that call. In Python and JavaScript the tools are async functions. A deadline stops the wait, not necessarily the tool, so make tools safe to abandon.

import asyncio
import json


async def run_call(registry, call):
    """Run one tool call. Always returns a tool message, even when the tool fails or is too slow.
    The registry maps a name to an async function (async def) that takes the parsed arguments."""
    fn = call["function"]
    tool = registry.get(fn["name"])
    try:
        if tool is None:
            content = json.dumps({"error": "unknown tool: " + fn["name"]})
        else:
            result = await asyncio.wait_for(tool(json.loads(fn["arguments"])), timeout=5)  # one deadline per tool
            content = json.dumps(result)
    except asyncio.TimeoutError:
        content = '{"error":"timed out after 5 s"}'
    except json.JSONDecodeError:
        content = '{"error":"arguments are not valid JSON"}'
    except Exception:
        content = '{"error":"the tool failed unexpectedly"}'  # log the real exception on the server (Topic 7.5)
    return {"role": "tool", "tool_call_id": call["id"], "content": content}


async def run_calls(registry, calls):
    # gather returns results in the order of the calls, whatever order they finish in.
    return await asyncio.gather(*(run_call(registry, c) for c in calls))

A real model asks for two tools at oncereal model

Try: Send as it is: this model asks for get_weather twice in one reply (two tool calls with different ids). Press 'Answer the tool call': it creates one tool message per id. Fill in both results and send again. Then reset, switch the model to openai/gpt-oss-20b and send: it asks for one city at a time, so the loop runs twice. Both are correct, and your loop must handle either.

System

user

Modelmid-size, also reads imagesTemperature · 0.0Max tokens (1–1024) Stream (SSE) JSON mode Tools: get_weather

Goals

The model calls get_weather

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Reuse one id

In the first example (Python), write "tool_call_id": "call_1" for every tool message instead of call["id"].

terminal

$ Run

── what you'll see ──

the model asked for 3 tools in one turn

call_1 get_weather {"city":"Pune"} -> {"city":"Pune","temp_c":31} (120 ms)

call_2 get_weather {"city":"Delhi"} -> {"city":"Delhi","temp_c":36} (150 ms)

call_3 get_stock {"symbol":"ACME"} -> {"symbol":"ACME","price":42} (300 ms)

one after another: 570 ms

all at once: 300 ms (the slowest call)

tool messages for: call_1, call_1, call_1

Myth vs fact

Myth

Results must be sent back in the order they finish.

Fact

Results are matched by tool_call_id. Send them in the order of the calls so the history reads naturally.

Myth

Parallel calls always make the answer faster.

Fact

They save time only when the calls are independent and the tools are slow. Dependent calls cannot be parallel, and tiny fast tools gain nothing.

Myth

The model returns one tool call per reply.

Fact

It may return none, one or several, along with text. Always loop over the list.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Bound concurrency: if a model asks for fifty calls in one reply, run at most a handful at a time and cap the number you accept (refuse or truncate beyond a limit and tell the model).
▸
A slow tool dominates: set a deadline per call and a deadline for the batch, and return a timeout error message for the late one so the others are not wasted.
▸
Concurrent writes need care. Use idempotency keys or per-record locks, or run write tools sequentially even if the model asked for them together.

Remember this

1
Why this exists. Some questions need several independent lookups: 'weather in Pune and Delhi and the ACME share price'. A model that can only ask one question per turn needs three round trips, each with a model call and its delay. Models can instead put several tool calls in one assistant message (a tool_calls list with several entries). You run them together and make one extra round trip instead of three.
2
What the reply looks like. The same tool_calls list, with more than one element, each with its own id. Your code must handle a list of any length (including one), never assume the first element is everything. You answer every id with a tool message (recall from Topic 7.2: each tool message repeats the tool_call_id). Leaving one unanswered makes many APIs reject the next request. Whether a model batches is its own choice: in the lab below, one model asks for both cities in one reply and another asks for one city per turn, so your loop must work for both.
3
Match by id, not by position. If you run calls concurrently they finish in any order. The tool_call_id on each tool message is what links result to request. The second example shows the bug: pairing results by arrival position gives Pune the result for Delhi, and the model then writes a confident wrong answer. Pair by id, and send the messages in the order of the original calls to stay on the safe side.
4
Time and cost. Run one after another, the total time is the sum of the tool times. Run together, it is about the slowest one. The first example shows 570 ms against 300 ms with simulated tool times. The tokens are the same either way. A provider setting such as parallel_tool_calls (in the OpenAI API and compatible ones; check your provider) can turn parallel calls off when your tools must happen in order.
5
Only independent calls run together. If the second call needs the first call's result (find the order, then fetch its shipment), the model cannot ask for both at once: it asks for the first, reads the result, and asks again. Concurrency is safe only for reads, or for writes that do not touch the same thing. Two calls that update the same record in parallel can clash, so for writes prefer sequential execution or use locks or idempotency keys (Topic 7.7).
6
Concurrency needs limits. Cap how many tool calls run at once (a thread pool, which keeps a fixed number of worker threads, or a semaphore, a counter that lets only N tasks in at a time, Topic 6.10), give each its own timeout, and make sure one failure becomes that call's error message, not a crash of the others. In Java this course's examples are single-threaded, so the Java example runs the calls in a loop and simulates the time. The code below the example shows the real concurrent versions in all three languages.

Explain it without notes

How does a model signal several tool calls in one turn, and how must the application answer?

Why is pairing results with calls by position a bug when tools run concurrently?

When should you not run tool calls in parallel?

Practice

Write pair_by_id(calls, results) where results is a list of (tool_call_id, content) in any order. Return the list of tool message dicts in the order of calls (each call is a dict with an id).

Write batch_time(times, parallel) that returns the total simulated milliseconds: the sum if parallel is false, the maximum if true, and 0 for an empty list.

Trade-offs

↔
Parallel execution cuts waiting time and needs concurrency control, timeouts and care with writes.
↔
Sequential execution is simpler and safer for writes and costs the sum of the tool times.
↔
Accepting many calls per turn lets the model batch its work and gives it a way to flood your tools; set a cap.

Done when you can

Done when your loop handles a tool_calls list of any length.
Done when every call gets exactly one tool message with its own id.
Done when results are paired by id, never by arrival position.
Done when concurrency, per-call timeouts and a cap on the number of calls are set.
Done when writes that could clash run one at a time.

Command Palette

Words you'll meet

Step by step

01Step 1: one reply, three calls

02Step 2: run them

03Step 3: results arrive in any order

04Step 4: pair by id

05Step 5: the real thing

Try it yourself

Change the times

Break the pairing

Count the messages

Code & diagrams

Break it on purpose

Reuse one id

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can