PHASE 12Advanced ~27 min· topic 8 of 8

Topic 12.8

Build: Mitra Works Offline for Simple Things

In one line

Mitra learns to answer the simple, common questions (returns, shipping, opening hours) on the device, with no network, and to send everything else to the server. The interface says where each answer came from and what is still possible offline.

Think of it like this

A shop assistant with a laminated card of the most common answers ("returns: 7 days", "opening hours: 9 to 6"). For those, they answer at once without calling anyone. For "where is my parcel" they phone the warehouse. If the phone line is down, they say: "I can tell you about returns and opening hours, but not about parcels right now." The card is the on-device part. The phone is the server. Mitra gets both.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Router: A function that decides which path a question takes: answered on the device, or sent to the server.
Intent: What the user wants, such as "returns" or "opening hours".
setMessages: A useChat function that replaces the message list. Used here to add a locally made exchange.
navigator.onLine: A browser flag. false means offline for sure; true only means a network interface exists.
Source label: A small tag on each answer saying "on this device" or "from server".
Graceful degradation: When a part stops working, the rest still does, and the user is told what is still possible.

Step by step

01Start from the previous Mitra

The route and the chat state are unchanged: serve("/api/chat", …) calls streamText and returns result.toUIMessageStreamResponse(); the component calls useChat with a DefaultChatTransport. Nothing about the server changes in this topic, which is the point: the device path is added in front of it.

02Write and test the router first

The first live example is only the router, as a function you can read in one screen. It checks, in order: empty input (ignore), an order number (server), no Latin words (server), then counts intents. Eight inputs print eight decisions.

Writing the router as a pure function first means you can test the decisions before the screen exists, and it is the piece you will later replace with a real classifier (Topic 12.2) without touching the screen.

localAnswer (the shape the live example uses)whole filejs

function localAnswer(question) {
  if (/ord-\d+/i.test(question)) return null;         // live order data: server
  const words = question.toLowerCase().match(/[a-z]+/g) || [];
  const hits = INTENTS.filter((i) => i.words.some((w) => words.includes(w)));
  return hits.length === 1 ? hits[0] : null;          // none or several: not sure, so no guess
}

03Add the device branch to submit

In the submit handler, after the usual guards (empty text, busy), call localAnswer. If it returns an intent, build two messages by hand, the user's and the assistant's, in the same parts shape useChat uses, and call setMessages([...messages, userMessage, assistantMessage]). Record sources[id] = "device". No request is made, and status never leaves ready.

For the server path nothing changes except a label: onFinish records sources[message.id] = "server". A stopped answer is labelled too: with useChat, stop() keeps the partial answer and still calls onFinish, with isAbort: true (Topic 13.5 counts stops this way). Only useObject (Phase 7) skips its callbacks on stop().

04Handle offline before sending

If the question needs the server and online is false, do not send. Show "You are offline. I can still answer questions about returns, shipping and opening hours." in the role="alert" line. This is an early, kind message; if online was wrong and the request fails anyway, the usual error handling from Phase 3 covers it.

Handle offline before sendingdiagram

Rendering diagram…

05Watch it run

The second example is Mitra with a scripted visitor. It asks about a refund (device), asks where ORD-1042 is (server), then goes offline and asks about opening hours (device, still works) and where ORD-1043 is (cannot be answered offline), then comes back online. The log shows each decision, the number of messages on screen, the offline message and the final list of labels: [on this device] [from server] [on this device].

Then try your own questions, including a two-topic question and Hindi text. The breakIt below shows what happens if you delete the offline rule.

Try it yourself

1
Teach the device a new answer
In the second example add an intent to INTENTS: { name: "gift", words: ["gift", "wrap", "wrapping"], answer: "Gift wrapping costs nothing and is added at checkout." }, then ask "Do you do gift wrapping?" while offline.
It answers on the device. The route, the labels and the offline rule did not change.
2
Ask two things at once
Ask "I want to return it and know the delivery time". The router finds two intents and sends it to the server.
Change hits.length === 1 to hits.length >= 1 and ask again: the device answers only the first topic and ignores the second. A confident half-answer is worse than a server call.

Code & diagrams

Step 1: the on-device router, with every decision tested

A stand-in router: keywords, not a neural network. Replace the body with a real small model later and keep the same returns: a device answer or "go to the server".

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

"Can I get a refund?" -> device: returns
"When will it ship?" -> device: shipping
"What are your opening hours?" -> device: hours
"Where is ORD-1042?" -> server (needs live order data)
"I want to return it and know the delivery time" -> server (unsure: returns and shipping)
"Tell me a story" -> server (no intent matched)
"मेरा ऑर्डर कहाँ है" -> server (no Latin words, the stand-in cannot read this)
"   " -> ignore (empty)

Mitra after Phase 12

A scripted visitor asks five questions across an online-offline-online round trip. The server model is a scripted fake that always says the same thing about ORD-1042. Real app: your route and a real model.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

answered on device: returns
messages on screen: 4
answered on device: hours
offline, cannot answer: Where is ORD-1043?
error shown: You are offline. I can still answer questions about returns, shipping and opening hours.
all sources: [on this device] [from server] [on this device]

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Remove the offline rule

In the second example change if (!online) { to if (false) {.

terminal

$ The end of the output

── what you'll see ──

error shown: undefined

all sources: [on this device] [from server] [on this device] [from server]

Myth vs fact

Myth

navigator.onLine === true means my server is reachable.

Fact

It only means the device has a network connection. The server can still be unreachable or slow, so handle request failures anyway.

Myth

An on-device answer needs no testing because it is local.

Fact

A router that answers wrongly and confidently is worse than one that asks the server. Test the decisions, especially the unsure ones.

Myth

Offline support means the whole app works offline.

Fact

It means the useful parts that do not need live data work, and the rest explains clearly why it cannot.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
The local answers are content, and content goes stale. A hard-coded "7 days" in the page is wrong the day the policy changes. Load the answers from the same source as the server's (a file shipped with each deploy, versioned like the model in Topic 12.4), and show them as the shop's official text, not as generated.
▸
Keep local exchanges in the saved conversation (Phase 4), with their source, or users will lose them on reload and the history will have gaps. The server may also want to know a local exchange happened if it builds context from the conversation.
▸
Add a way out of a wrong local answer. A small "Ask the assistant instead" link under device answers sends the same question to the server. It costs a line of code and removes the worst failure of a router: a user stuck with an answer that is close but wrong.
▸
Count the share of questions answered on the device and the share where users then ask the same thing again. That is the quality measure of the router (Phase 13).

Remember this

1
The problem this solves: Mitra sends every question to the server, even "how many days do I have to return something". That costs a request, takes the network's time, and fails completely when the user is offline. This topic adds a small on-device router that answers the common questions locally and sends the rest on, so the common case is instant and survives bad connections.
2
Where Mitra starts. After Phase 11, Mitra is a useChat chat (Phase 5) with a /api/chat route that uses streamText, safe rendering (Phase 6), cards and tools (Phases 7 and 8), safety (Phase 9), product polish and accessibility (Phase 10), and images and voice (Phase 11). To keep this example readable, the live version keeps only the chat core: the route, useChat, #messages, #prompt, #send, #stop, #status and #error. Everything from earlier phases still applies to the full app.
3
What is added. (1) A localAnswer(question) function that decides whether the device can answer. (2) A branch in the submit handler: if there is a local answer, add the user's message and the answer to the chat directly with setMessages, without calling the server. (3) A record of where each answer came from, shown as a small label. (4) An offline rule: if the browser is offline and the question needs the server, show a helpful message instead of a failing request.
4
The router has to be careful. It answers only when exactly one intent matches. Zero matches means it does not know, so the question goes to the server. Two matches means it is unsure ("I want to return it and know the delivery time"), so the question goes to the server. A question with an order number (ORD-1042) always goes to the server, because order status is live data the device cannot know. Non-Latin text finds no words and also goes to the server. The first live example prints every one of these decisions.
5
Offline. The page knows it is offline from navigator.onLine and the online and offline events. navigator.onLine is only a hint: false is reliable (there is no network), but true only means a network interface exists, not that your server is reachable. So the code treats a failed request like any other failure (Phase 3) and uses the offline flag only to give a better message early. In the live example, a prop online stands in for it, because the sandbox cannot go offline.
6
What Mitra can do now that it could not before: answer returns, shipping and opening-hours questions with no network and no server cost; say whether an answer came from the device or the server; and tell an offline user exactly what is still possible. What is still missing, and which phase adds it: a usage meter, daily limits, fallbacks between servers, metrics and tests (Phase 13); and long-running, multi-step agent work (Phase 14). The real on-device model for classification and the hybrid with Chrome's built-in AI (Topics 12.2 and 12.7) plug into localAnswer without changing anything else.

Explain it without notes

How does Mitra decide whether to answer on the device or ask the server?

Why does the router answer only when exactly one intent matches?

What does Mitra do offline, and why is navigator.onLine only a hint?

How would you replace the keyword router with a real on-device model?

Practice

Add an "Ask the assistant instead" button under each device answer that sends the same question to the server and replaces nothing: the server's answer is added below.

Persist sources along with the messages in localStorage under a versioned key, and restore both on load, ignoring data from an older version.

Trade-offs

↔
Answering common questions on the device makes them instant and free, but the answers live in two places (device and server) that must be kept in step. Keeping one source of truth and shipping it to both costs build work but avoids contradictions.
↔
A cautious router (exactly one match) sends more to the server and answers fewer on the device. A bolder one saves more but gives more wrong answers. Choose the cautious one for facts that matter, like returns policy.

Done when you can

I can explain the router's decisions and test them as a function.
I can add a locally made exchange to a useChat conversation with setMessages.
I can label each answer with where it came from.
I can give a useful message when the user is offline and the question needs the server.
I can say what Mitra can do now and what Phase 13 and Phase 14 add.

Command Palette

Words you'll meet

Step by step

01Start from the previous Mitra

02Write and test the router first

03Add the device branch to submit

04Handle offline before sending

05Watch it run

Try it yourself

Teach the device a new answer

Ask two things at once

Code & diagrams

Break it on purpose

Remove the offline rule

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can