Frontend System Design: Design a Chat UI

01Step 1: requirements and numbers

Write two short lists. Functional: type a question and see a streamed answer; stop it; retry or regenerate; keep a history of the conversation; show answers with formatting and code; show an order card and ask before cancelling an order. Non-functional: first words in about a second on a normal connection; works on a phone; a conversation of 200 messages stays smooth; no key in the browser; accessible with a keyboard and a screen reader; model output is treated as untrusted; daily cost is bounded.

Then say what is out of scope for version one: voice, branching, attachments, several agents. Tell the interviewer you will return to them at the end.

02Step 2: the protocol

Choose: the page sends a POST with the conversation to your route, and the route answers with a streamed text/event-stream response. The page reads it with fetch (Topics 2.2 to 2.5, or the AI SDK's transport in Topic 5.3). Stop is abort() on that request (Topic 3.1).

Alternatives and why you do not choose them: polling adds delay and wastes requests; a WebSocket is two-way and keeps a connection open per user, which costs more to run and needs its own reconnect and auth logic. A chat answer is one request and one long reply, which an HTTP stream already does well. A WebSocket becomes reasonable when the server must push things unrelated to a request (a long-running agent that finishes while you are away, Topic 14.2).

03Step 3: state, and where each piece lives

Split state into three homes. On the screen: the text in the box, whether the answer panel is scrolled to the bottom, which approval card is open. In the browser: the draft, a cached copy of recent conversations so the page opens instantly. On the server: the conversations that matter (account-owned, searchable, shared between devices), the usage counters and the audit log.

Inside the page, keep one reducer with a list of messages (stable ids, a role, parts, a status) and one status for the whole request. Say the events: send, token, done, error, stop. The reducer ignores events for an old request id (Topics 4.1, 4.2 and 4.7). Persist after each completed turn, not after each token.

04Step 4: rendering a long, growing conversation

Three problems and their answers. Re-rendering every message on every token: give each message a stable id key and wrap the row in memo, so only the streaming message changes. Parsing Markdown again for the whole answer on every token: split the answer into blocks and parse only the last, unfinished block (Topic 6.5). Thousands of rows: virtualise, drawing only the rows near the screen (Topic 6.5), and keep the scroll position when new rows arrive above or below.

The first live example counts how many row components render while 13 tokens stream into a list of 50 messages. Read the three numbers: without memo every row renders on every token; with memo almost none do; and memo with a fresh copy of each message on every update is no better than no memo at all, a mistake that is easy to make.

05Step 5: failure, one table

List every failure and give each its own message, its own retry rule and a decision about the partial answer. The second live example is that table as one function. It takes what happened (a status, offline, a broken stream, a stop, a content filter, a length cut) and returns the sentence for a person, whether a retry makes sense, and whether to keep the partial text. Raw provider errors never reach the screen (Topics 1.3 and 3.3).

06Step 6: scale

Long chats: every request resends the history, so cost and delay grow with every turn. Set a context budget, keep the newest turns that fit, tell the user when older turns are no longer sent, and refuse a single message that alone is too big instead of cutting it (Topic 4.3). The third live example does this and prints what happens at each edge.

Many users: the server counts usage per user with an atomic check, answers 429 with retry-after, and the page shows a countdown (Topics 3.6 and 13.2). Many tabs: a browser storage event keeps drafts in sync, and the server is the source of truth for the history. Slow starts and breaks: a watchdog for a stalled stream (Topic 3.5) and a resume for long jobs (Topic 14.2).

07Steps 7 and 8: safety and accessibility

Safety: output is untrusted, rendered as React elements and never as HTML; links and images follow an allow-list; tools that change things need an approval card built from server data, and every tool call is checked against the signed-in user on the server (Topics 6.3, 8.6, 9.5).

Accessibility: one polite live region that announces when an answer starts and when it is complete, not every token (Topic 10.6); focus moves to the approval card when it appears and back to the box afterwards; Stop and Copy work with the keyboard; motion respects prefers-reduced-motion; code blocks keep sufficient contrast. Finish by saying what you would measure: time to first token, stop rate, error rate, feedback (Topic 13.5).