PHASE 13Advanced ~29 min· topic 6 of 8

Topic 13.6

Testing AI UI

In one line

Test the screen, not the model: replace the model with a script, drive the component like a user, wait for what appears on screen instead of for timers, and cover every state (waiting, streaming, done, 429, broken stream, stopped).

Think of it like this

A fire drill. Nobody sets a real building on fire to check the exits. You rehearse with a script: "alarm now, door three blocked, one person limping". The drill checks that people and exits behave correctly in each situation. Testing an AI screen is a drill: script the model's behaviour, and check what the screen does.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Mock: A stand-in for a real part (a model, a network call) that answers from a script.
Testing Library: Tools to render a component and find things on screen the way a user does.
findBy / waitFor: Testing Library helpers that retry until something appears or a timeout passes.
Flaky test: A test that sometimes passes and sometimes fails with no code change, usually because of timing.
Component test: A test that renders one component (and its children) and checks what the user would see.
End-to-end test: A test that runs the whole app in a real browser.
Stream timeline: The list of events a route produces, with repeats collapsed, used as a snapshot.
Cleanup: Removing the rendered component after a test so tests cannot affect each other.

Step by step

01Script the model per test

In the first live example the route reads a variable script. A test sets it before rendering: a plain string for a normal answer, { fail: { status: 429 } } for a busy provider, { text: "Orders ship in", error: "connection reset" } for a stream that breaks after three words. The component is the same in every test.

02Write the test like a user story

"Type a question, press Send, expect Thinking…, then the answer." In code: user.type, user.click, await screen.findByText("Thinking…"), await screen.findByText("Orders ship in two days."). Nothing about timers, state or class names.

The error tests read the screen the way a screen reader would: findByRole("alert") and check its text. If the component forgot role="alert", the test fails, and so would a blind user's experience.

03Stop in the middle of a stream

The Stop test cannot use a timer. It waits for the first word (await screen.findByText(/Orders/)), then clicks Stop, then waits for the label "Stopped". The test acts on an event of the stream, so it behaves the same on a slow laptop and a fast one.

04When a test fails

The last test is wrong on purpose: it waits 400 ms for words the page never shows. findByText keeps retrying, then fails with "Unable to find an element with the text…" followed by a long hint and a dump of the page, which this example trims to the first sentence. In a real project the dump is the most useful part: it shows what *was* on screen.

05Snapshot the timeline

The second example records the event types of a stream. A normal answer is start > start-step > text-start > text-delta x5 > text-end > finish-step > finish. A broken stream ends with error > finish-step > finish. The third expectation is stale on purpose: nobody updated it when reasoning parts started to arrive, and the test prints expected and actual so the change is easy to see.

It ends with a shape check on three different replies. The third reply contains an HTML tag and fails: your tests should assert what the UI depends on, not the model's wording.

06The same test in a real project

In a project you would write this with Vitest. The test bodies are almost the same as the live ones; the difference is the file, the runner and the way the model is scripted.

chat.test.tsx (Vitest, not run here)whole filetsx

// vitest.config.ts: test: { environment: "jsdom", globals: true, setupFiles: ["./test/setup.ts"] }
// test/setup.ts:    import "@testing-library/jest-dom/vitest";
import { render, screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { MockLanguageModelV4 } from "ai/test";
import { vi, test, expect } from "vitest";
import { Chat } from "@/components/chat";

const scripted = (text: string) => new MockLanguageModelV4({
  doStream: async () => ({
    stream: new ReadableStream({
      start(c) {
        c.enqueue({ type: "stream-start", warnings: [] });
        c.enqueue({ type: "text-start", id: "t" });
        c.enqueue({ type: "text-delta", id: "t", delta: text });
        c.enqueue({ type: "text-end", id: "t" });
        c.enqueue({ type: "finish", finishReason: { unified: "stop", raw: "stop" }, usage: { inputTokens: { total: 1 }, outputTokens: { total: 1 } } });
        c.close();
      },
    }),
  }),
});

test("shows the streamed answer", async () => {
  // doMock only affects modules imported AFTER it, so import the route inside the test
  vi.resetModules();
  vi.doMock("@/lib/model", () => ({ model: scripted("Orders ship in two days.") }));
  const { POST } = await import("@/app/api/chat/route"); // your real route, now using the scripted model
  vi.stubGlobal("fetch", (url: string, init: RequestInit) => POST(new Request("http://localhost" + url, init)));
  const user = userEvent.setup();
  render(<Chat />);
  await user.type(screen.getByLabelText("Your question"), "How long is shipping?");
  await user.click(screen.getByRole("button", { name: "Send" }));
  expect(await screen.findByText("Orders ship in two days.")).toBeInTheDocument();
});

Try it yourself

1
Add a test for an empty reply
In the first example add a test where script = "". Ask, then check what the page shows. With an empty model reply this Chat shows nothing at all: no "Thinking…", no answer, no error, and the box is back to normal. A shopper cannot tell whether Mitra heard them. Decide what it should say (for example "Mitra had nothing to say. Please rephrase."), write the test first, then make it pass.
2
Cut the answer at the length limit
Set script = { text: "Tea began in China and then", finish: "length" }. The page shows the text with no sign it was cut. Add the label "This answer was cut off by the length limit" to Chat: make the route send messageMetadata with the finish reason as in Topic 13.1, show the label from message.metadata, and make the test pass (see the practice answer).
3
Break a timing assumption
Change firstDelayMs from 150 to 30 in the route and run again. All tests still pass, because they wait for the screen, not for a number of milliseconds. The sleep version from the break-it needed a different number for each delay: it had to be bigger than the whole answer time, and that time is exactly what you changed.

Code & diagrams

Six tests against a scripted model, and one that fails

The component is the unit under test and the scripted model is the only fake. The last test fails on purpose, to show what a failure looks like. Testing Library really prints much more: a hint and the page as text.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

PASS Send is disabled until there is text
PASS shows Thinking… and then the streamed answer
PASS a 429 shows the busy message
PASS a broken stream keeps the partial answer and says it was cut off
PASS Stop ends the answer and says Stopped
FAIL this test is wrong on purpose: it expects words the page never shows → Unable to find an element with the text: Delivery takes a week..

Snapshot the stream, and test the shape instead of the wording

The stand-in produces the same events every time, so the timeline is a stable snapshot. With a real provider you would snapshot the events of a recorded, scripted stream, not of a live call.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

PASS a normal answer
PASS a stream that breaks
FAIL an answer with reasoning
  expected: start > start-step > text-start > text-delta x4 > text-end > finish-step > finish
  actual:   start > start-step > reasoning-start > reasoning-delta x3 > reasoning-end > text-start > text-delta x4 > text-end > finish-step > finish
PASS shape check: "Orders ship in two days."
PASS shape check: "Shipping takes 1 to 2 working days. Anything else?"
FAIL shape check: "<img src=x> Two days."

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Wait with a timer

In the first example replace the line await screen.findByText("Orders ship in two days."); of the second test with await new Promise((r) => setTimeout(r, 50)); expect(Boolean(screen.queryByText("Orders ship in two days."))).toBe(true);.

terminal

$ Console, second line

── what you'll see ──

FAIL shows Thinking… and then the streamed answer → expected true, got false

Myth vs fact

Myth

AI features cannot be tested because the model is random.

Fact

The model is random, your screen is not. Script the model and the screen becomes deterministic. What is left to measure about the model belongs to evals.

Myth

A longer sleep makes a test safer.

Fact

It makes the test slower and still guesses. Waiting for a visible result is both faster and stronger.

Myth

Snapshots of the DOM are enough.

Fact

Big DOM snapshots change for harmless reasons and get approved without reading. Small, hand-written expectations about states and stream events say what you actually care about.

Myth

Mocking the model means the test proves nothing.

Fact

It proves the screen handles each model behaviour. It never claimed to prove the model is good.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Keep a library of scripts shared by tests, evals and Storybook-style demos: normal, slowStart, breaksAfter3Words, rateLimited, refusal, emptyReply, cutOffByLength, hugeAnswer, emoji. A new state in production becomes a new script plus a test.
▸
Test the unhappy path of the same component that ships. A separate test-only component proves nothing.
▸
Run a few end-to-end tests against the production build with a mocked route (for example Playwright with route interception), because component tests do not see bundling, headers or streaming through a real server and CDN.
▸
Keep a test that checks accessibility roles for the status messages (alert, status) and the focus after sending, since AI screens update without the user doing anything.

Remember this

1
The problem this solves: A real model is slow, different every time and costs money, so a test that calls it is slow, flaky and expensive. And the interesting bugs live in rare states: a 429, a stream that breaks after three words, a Stop at the wrong moment. You cannot make a real model do those on demand. A scripted model can. This topic shows how to test every state quickly and the same way every time.
2
Test the screen, not the model. Your code decides what the user sees for each model behaviour. Test that decision. Do not assert the exact words a real model writes, because they change. Assert the *shape* (non-empty, not too long, no raw HTML) and let evals (Topic 13.7) watch quality.
3
Script the model at the edge. Two common places: replace the model (on the server side, a mock model such as MockLanguageModelV4 from ai/test, which is what this course's stand-in is built on) or replace **fetch** in the browser so it returns a prepared response. Either way the component runs unchanged. The live example uses a route whose model is scripted per test: a normal reply, a 429 failure, a reply that breaks after "Orders ship in", and so on.
4
Drive it like a user. Testing Library's render, screen and userEvent find elements the way a person would: by label (getByLabelText("Your question")), by text, by role (getByRole("alert")). If a test cannot find something by what the user sees or hears, the screen is probably not accessible either. Avoid reaching into component state or CSS class names.
5
Wait for the screen, never for the clock. Streams take time, and the tempting fix is await sleep(500). It is too slow when it is long and flaky when it is short (the last live example shows a 50 ms sleep failing). Use findByText(…) or waitFor(() => …), which retry until the thing appears or a timeout passes. For actions in the middle of a stream, trigger them from what appeared: await findByText(/Orders/), then press Stop. This is the same rule as for the course's own examples.
6
Cover every state. The list for a chat: empty and disabled Send, waiting ("Thinking…"), streaming, done, error with a message per cause (429, 500, offline), a broken stream with the partial text kept, Stop and the "Stopped" label, a cut-off answer (finish reason length), a refusal, an empty reply. Each is a small test with a script. The first live example runs six.
7
Snapshot the stream, not the pixels. A stream timeline is the list of events a route produces, collapsed into runs (text-delta x5). Store the expected list in the test. A changed timeline means the protocol your page depends on changed, and the diff shows exactly where. Keep snapshots small and written by hand, and review every change.
8
Where each kind of test belongs. Reducers and helpers (pure functions): plain unit tests. The chat component with a scripted model: component tests with Testing Library. One or two whole-page flows in a real browser with a mocked route: end-to-end tests (for example Playwright). Answer quality: evals. The preview here has no Vitest, so the example has a 12-line test() and expect(); a real project uses Vitest (below), and the test bodies are the same.

Explain it without notes

Why script the model instead of calling a real one in tests?

How do you wait for a streamed answer in a test, and why not with a timer?

How do you test a Stop button without a flaky test?

What is a stream timeline snapshot and when would you use one?

Practice

Write a test for the 'cut off by length' case: script = { text: "Tea began in China and then", finish: "length" }. The page should show the text and a label "Cut off by the length limit".

Write a test that a second question is not sent while the first is still streaming: the Send button must be disabled until the first answer is complete.

Trade-offs

↔
Component tests with a scripted model are fast and cover every state, and they do not see the real network, the bundler or the real provider. A few end-to-end tests cover that, at the price of being slower and more fragile.
↔
Hand-written stream snapshots catch protocol changes and need updating on purpose when the SDK changes. That friction is the feature.

Done when you can

I can script a model per test, including failure and broken-stream behaviours.
I can write a Testing Library test that waits for what appears instead of a timer.
I can test Stop by acting on a stream event.
I can list the states of a chat that each need a test.
I can snapshot a stream timeline and explain why the model's wording is not asserted.

Command Palette

Words you'll meet

Step by step

01Script the model per test

02Write the test like a user story

03Stop in the middle of a stream

04When a test fails

05Snapshot the timeline

06The same test in a real project

Try it yourself

Add a test for an empty reply

Cut the answer at the length limit

Break a timing assumption

Code & diagrams

Break it on purpose

Wait with a timer

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can