PHASE 8Advanced ~25 min· topic 5 of 9

Topic 8.5

Client-Side Tools

In one line

Some actions can only happen in the browser: opening a panel, reading the clipboard, asking for the user's location. A tool declared without execute is handed to the browser, which runs it, reports the result with addToolOutput, and the chat is sent again so the model can continue.

Think of it like this

A phone operator who says "please read me the number on the card in your hand". The operator cannot see your hand. Only you can. So the operator asks, you do it, you read the number back, and the conversation continues. A client-side tool is a request that only the user's device can carry out.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Client-side tool: A tool the browser runs. It is declared without execute on the server.
onToolCall: A useChat option called when a tool call arrives, where the browser does the work.
addToolOutput: The useChat function that gives a tool call its result (or its error) from the browser.
sendAutomaticallyWhen: A useChat option: a function that says when to send the conversation to the server again.
lastAssistantMessageIsCompleteWithToolCalls: The ready-made condition: every tool call in the last step has an output.
Permission prompt: The browser's own question "Allow this site to use your location?". The answer is remembered.
UI action: A tool whose job is to change the page: open a panel, highlight a row, scroll.

Step by step

01Two round trips, one answer

A server tool finishes inside one request. A browser tool needs two. First request: the model asks, the stream ends with the tool part waiting. The browser runs the tool. Second request: the browser sends the conversation again with the result, and the model answers.

Two round trips, one answerdiagram

Rendering diagram…

02A UI action

The first live example declares showOrder with no execute. In the browser, onToolCall sets React state (the panel now shows ORD-1042) and gives the result { opened: "ORD-1042" }. sendAutomaticallyWhen resends, and the model says "I opened ORD-1042 for you."

The console shows both requests. The server sees one message first, then two. The (browser) line shows where the tool really ran.

03A tool the user can refuse

The second example asks for the user's location. It does not call the browser straight away: it shows a card, and Not now reports an error result, The customer chose not to share their location.. The model reads it as the tool's result and asks for a pincode instead.

The example presses Not now twice on purpose. The second press does nothing: the first answer removed the card from the page. If your card could still be pressed after the answer, guard against it.

04What the real location call looks like

In a real app, Allow would run the code below. Browsers answer in three ways: success, a refusal (code 1), or a failure (code 2 position unavailable, code 3 timeout). Feature-detect first: navigator.geolocation may not exist, and the page must be served over HTTPS. (In this site's sandboxed preview the object exists, but the call is blocked by a permissions policy and answers with code 1. The clipboard is the same: navigator.clipboard.writeText is rejected with NotAllowedError. That is why the working path is shown here and not run.)

allow.ts (real app, not run here)whole filets

function getLocation(): Promise<{ lat: number; lon: number }> {
  return new Promise((resolve, reject) => {
    if (!("geolocation" in navigator)) return reject(new Error("This browser cannot share a location."));
    navigator.geolocation.getCurrentPosition(
      (pos) => resolve({ lat: pos.coords.latitude, lon: pos.coords.longitude }),
      (err) => reject(new Error(err.code === 1 ? "The customer blocked location access." : "Could not get a location.")),
      { timeout: 8000 },
    );
  });
}

// in onToolCall, after the customer pressed Allow:
// getLocation().then(
//   (output) => addToolOutput({ tool: "getLocation", toolCallId, output }),
//   (e) => addToolOutput({ tool: "getLocation", toolCallId, state: "output-error", errorText: e.message }),
// );

Try it yourself

1
See the two requests
In the first example change the console line in the server route to also print messages.map((m) => m.role).join().
Request 1 prints user; request 2 prints user,assistant. The second request carries the assistant message with the finished tool part.
2
Say yes to location
In the second example add a second button Allow that calls addToolOutput({ tool: "getLocation", toolCallId: ask.toolCallId, output: { city: "Pune" } }).
Press it by hand instead of Not now. The model sees an object, not text, and answers "Delivery to your area takes 2 days."

Code & diagrams

A tool that runs in the browser

The first request ends with the tool waiting for the browser. The second request carries the result. Both are visible in the (server) lines.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

(server) request arrives with 1 messages
(browser) running showOrder {"orderId":"ORD-1042"}
(server) request arrives with 2 messages
page: Orders panel: ORD-1042 | showOrder: output-available | I opened ORD-1042 for you.

The customer says no

The card is Mitra's own, shown before any browser prompt. Pressing Not now twice sends one answer: the card is gone after the first press.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

(server) request arrives, last part: text
card open: true
(server) request arrives, last part: step-start,tool-getLocation/output-error
card open: false | answer: No problem. What is your pincode, so I can check delivery?

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Forget sendAutomaticallyWhen

In the first example delete the line sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls, and press Run.

terminal

$ Console

── what you'll see ──

(server) request arrives with 1 messages

(browser) running showOrder {"orderId":"ORD-1042"}

Myth vs fact

Myth

A client-side tool runs on the server when the browser is offline.

Fact

Nothing can run it except the browser. With no execute and no answer from the browser, the call just waits.

Myth

addToolOutput sends the result to the server.

Fact

It only stores the result in the message. Sending is sendAutomaticallyWhen's job (or your own sendMessage).

Myth

If the user blocks location, the tool call is cancelled by the SDK.

Fact

The SDK does not know. You must report it with an output-error result, or the call stays open.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Never trust a browser tool's result as a fact about the world on the server. It comes from the user's device, which the user controls. Use it to personalise ("near Pune"), never to authorise ("this person is the order's owner").
▸
onToolCall can run for several parallel tool calls. Answer each by toolCallId. sendAutomaticallyWhen waits until all calls in the step have an output.
▸
A page reload while a browser tool is waiting loses the in-memory conversation unless you saved it. Persist messages, and on load answer or cancel open tool calls so they do not hang.
▸
Clipboard writes need a user gesture, and reads need permission. Some environments forbid them altogether (embedded frames, insecure pages). Feature-detect, then report output-error with a clear text.
▸
Bound the loop yourself: count how many times the same tool has been answered in this chat and stop answering after a small number, because stopWhen on the server cannot see across requests.

Remember this

1
The problem this solves: Server tools can reach your database, but not the person's screen, their clipboard, their location or their camera. If the assistant should open an order panel, copy a tracking number or check delivery near the user, the browser has to do it. The model's request must travel to the browser, and the result must travel back.
2
How to declare one. Define the tool on the server with a description and an inputSchema, but **no execute**. The SDK cannot run it, so it ends the step with the tool part in the input-available state and sends it to the browser. The model still sees a normal tool.
3
How the browser runs it. useChat has an onToolCall option, called when a tool call arrives. Inside it you do the work, then call addToolOutput({ tool, toolCallId, output }). Two rules matter. First: **do not await addToolOutput inside onToolCall**. The SDK's own type comment says to call it without awaiting inside this callback (the callback's return value is not used). Second: always answer. A tool call with no output stays in input-available and the chat waits forever.
4
Sending the result back. Adding an output changes the message, but nothing is sent to the server by itself. You must tell useChat when to resend with sendAutomaticallyWhen. The ready-made lastAssistantMessageIsCompleteWithToolCalls says "resend when every tool call in the last step has an output". Then the server runs the model again with the result and the model writes the answer.
5
Permission and refusal. Some browser features need the person's permission: location, camera, microphone, clipboard reading. They can say no, or the feature may not exist (an old browser, an insecure page, a locked-down embedded frame). Report that to the model with state: "output-error" and an errorText, and let the model carry on ("No problem. What is your pincode?"). Never leave the tool hanging.
6
Ask before you use a sensitive feature. Do not trigger the browser's permission prompt just because the model asked. Show your own card first ("Mitra would like to use your location to check delivery. Allow / Not now"), and only call navigator.geolocation if they allow. A permission the user blocks in the browser is hard to undo, so spend it only when they expect it.
7
A loop to watch for. sendAutomaticallyWhen resends the conversation whenever its condition becomes true. Server-side stopWhen counts steps only inside one request, so it cannot stop a client that keeps resending. If a model keeps asking for the same browser tool, the browser keeps answering and resending. Limit it yourself (count resends, or stop answering the same call twice).

Explain it without notes

How do you make a tool run in the browser instead of on the server?

Why do you need sendAutomaticallyWhen for client-side tools?

The user denies the permission your tool needs. What do you do, and what happens if you do nothing?

Why should you show your own card before triggering a browser permission prompt?

Practice

Add a client tool copyTracking that copies "TRK-8841" to the clipboard and reports success or failure. Write the onToolCall branch.

Make the first example stop answering a browser tool after 3 resends in one chat.

Trade-offs

↔
Browser tools reach the page and the device, at the price of an extra round trip and a result that cannot be trusted on the server.
↔
Letting sendAutomaticallyWhen resend automatically is convenient and can loop. Resending yourself is more code and gives you control.

Done when you can

I can declare a tool with no execute so it runs in the browser.
I can handle it with onToolCall and addToolOutput without awaiting.
I can resend with sendAutomaticallyWhen.
I can report a refusal or missing feature as an output-error.
I know why browser results are never used to authorise actions.

Command Palette

Words you'll meet

Step by step

01Two round trips, one answer

02A UI action

03A tool the user can refuse

04What the real location call looks like

Try it yourself

See the two requests

Say yes to location

Code & diagrams

Break it on purpose

Forget sendAutomaticallyWhen

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can