PHASE 9Advanced ~23 min· topic 6 of 8

Topic 9.6

Refusals, Filters and Moderation in the UI

In one line

Sometimes an answer is stopped on purpose: the model declines, a provider filter cuts it off (finish reason content-filter), or your own moderation blocks the request. Each needs its own calm, honest message, a way to ask for a second look, and care for a half-written answer.

Think of it like this

A shop assistant who says "I'm sorry, I can't help with that here, but our manager can." The customer is not accused of anything. They know what happened and what to do next. Compare an assistant who goes silent, or one who shouts "VIOLATION". A refusal screen has the same job: stay polite, say what happened, and give a next step.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Refusal: A model answer that declines to help. It arrives as normal text.
Content filter: A provider-side check that can end a response. It reports finish reason content-filter.
Finish reason: The reason a response ended: stop, length, content-filter, tool-calls, error or other.
Moderation: Checking text against rules (yours or a service's) before or after the model.
False positive: Something harmless that was blocked by mistake.
Appeal: A way for the user to say "this block was a mistake" so a person can review it.
Whole-word match: A match that only counts when the word is not part of a longer word.
Partial answer: The text that arrived before the response was stopped.

Step by step

01Read the finish reason, not only the text

The first example runs the SDK against a scripted model four times: a normal answer, a filter that fires after two sentences, a filter that fires before any text, and an answer cut by the length limit. Each run reads the finish event from fullStream and chooses the screen state from the reason.

The filter is a stand-in: the scripted step carries finish: "content-filter". In a real app the provider sends it, and the SDK reports the same finishReason.

02Pick the message for each case

The function describe(reason, partial) returns the text the user sees and whether to keep the partial answer. A filter with partial text keeps it and marks it. A filter with no text shows the message alone. length says the answer was too long and offers "continue" (Topic 3.3).

Pick the message for each casediagram

Rendering diagram…

03Moderate the request before the model

The second example is a route that checks the request first. It returns 422 with a code the client understands, and the client shows a short, calm message with a "This was a mistake" button. The word list uses a harmless stand-in term, grumble.

It matches whole words with Unicode-aware boundaries, so grumbled and a Hindi sentence are not flagged, while Grumble! is. An empty message is refused for a different reason (nothing to send).

04Send the appeal without the text

Pressing the appeal button posts { messageId, reason } to a feedback route. The user's text is not included by default. A reviewer can open the stored conversation if the user consented, or ask the user to paste it. This keeps the appeal useful and the log clean.

Try it yourself

1
Match inside words
In the second example, change wholeWord so it returns new RegExp(word, "i"). Run it.
she grumbled about the tea is now blocked too. Substring matching is how innocent customers get told off.
2
Handle the missing reason
In the first example, change describe so it only checks reason === "stop" and treats everything else as an error with the message Something went wrong.
The filtered answers now look like crashes. Users cannot tell a policy stop from a bug, and neither can your support team.

Code & diagrams

Four ways a response ends

The filter is a stand-in: the scripted step says `finish: "content-filter"`. A real provider decides this itself. Notice that every case ended with the stream closing normally, so only the finish reason tells them apart.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

normal: reason=stop | kept text="Returns are accepted within 7 days." | banner="" | feedback button=false
filter, mid-answer: reason=content-filter | kept text="Returns are accepted within 7 days. Also, here is something the filter dislikes" | banner="This answer was stopped by our safety filter. If you think that was a mistake, tell us." | feedback button=true
filter, before any text: reason=content-filter | kept text="" | banner="This answer was stopped by our safety filter. If you think that was a mistake, tell us." | feedback button=true
too long: reason=length | kept text="Returns are accepted within 7 days. The full policy has many parts" | banner="The answer was long and was cut off. You can ask Mitra to continue." | feedback button=false

Moderating a request and appealing

Only the whole word is blocked: `grumbled` and the Hindi sentence pass. The appeal report holds an id and a reason, not the user's words.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

"Where is my order?" -> Happy to help with that.
"Grumble!" -> We can't help with that request here. You can rephrase it. [This was a mistake]
"she grumbled about the tea" -> Happy to help with that.
"ग्रम्बल की चाय कहाँ है?" -> Happy to help with that.
"   " -> nothing to send
appeal reports: [{"messageId":"msg-7","reason":"moderated"}]

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Block by substring

In the second example, replace the body of wholeWord with return new RegExp(word, "i"); (keep the signature) and run it.

terminal

$ Console, third message

── what you'll see ──

"she grumbled about the tea" -> We can't help with that request here. You can rephrase it. [This was a mistake]

Myth vs fact

Myth

If the stream closed normally, the answer is complete.

Fact

A filter or a length limit also closes the stream normally. Read the finish reason.

Myth

A refusal is an error and should show a red error box.

Fact

A model refusal is a valid answer. A filter stop is a policy outcome. Neither is a crash, and neither should look like one.

Myth

Longer block lists are safer.

Fact

Lists grow false positives faster than they stop determined users. Use them as one signal beside real moderation and a way to appeal.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Keep the safety stop state separate in your chat reducer: status: "stopped", stopReason: "content-filter". It then renders its own banner, offers its own actions and is counted in its own metric, instead of hiding inside a generic error state.
▸
When you moderate replies yourself (for example to remove a banned phrase), do it on the server before the text is streamed, or accept that a streamed word cannot be taken back. Streaming first and moderating later means the user already read it.
▸
Match the tone of your refusal message across all cases. If the wording changes depending on what was blocked, users learn what the filter looks for and probe it. A consistent message plus a feedback channel is better.

Remember this

1
The problem this solves: Safety systems sometimes stop an answer, and they are sometimes wrong. A frozen screen, a raw error such as content_filter, or a blaming message makes the product feel broken and the user feel accused. This topic covers the three ways an answer gets stopped and how the interface should treat each.
2
Three different things look alike. (1) A model refusal: the model writes text like "I can't help with that", and the finish reason is a normal stop. To your code it is just an answer. (2) A provider filter: the provider ends the response and reports finish reason content-filter (spelled content_filter in the OpenAI-style protocol). (3) Your own moderation: your server checks the request or the reply and refuses before or after the model. Each has its own signal and its own message.
3
The finish reason is the signal for filters. The AI SDK gives finishReason as one of stop, length, content-filter, tool-calls, error, other. Never decide a response is fine because the stream ended. Check why it ended. A length ending means the answer ran out of room (Topic 3.3), which is a different message from a filter.
4
A filter can fire mid-answer. The first sentences may already be on screen when the provider stops. Do not delete them silently, and do not leave them looking complete. Keep the text, mark it as stopped ("This answer was stopped because it may break the content rules"), and consider hiding it behind a "show what was written" control. If the filter fires before any text, show only the message.
5
Write respectful messages. Say what happened in plain words, say it was automatic ("our safety filter"), avoid accusing ("you tried to..."), say what the user can do (rephrase, ask something else, contact support), and never show internal codes. Keep it short and the same tone for every case, so users do not learn to game it by the reaction.
6
Offer an appeal and collect feedback. Filters have false positives. Add a "This was a mistake" button that sends an event with the message id, the reason and an optional comment, without copying the user's text unless they agree (Topic 9.7). Review a sample of these regularly; they are your best data about whether the filter is too strict.
7
Your own input moderation: match whole words. A simple word list is a common first step, and it has a famous failure: blocking innocent words that contain a blocked word (the "Scunthorpe problem", named after a town whose name was blocked by a filter). Use whole-word matching with Unicode-aware boundaries (so it also works in Hindi and other languages), and treat the list as one signal. For stronger checks, call a moderation service from your server.
8
Detect and verify. Detect: count filtered, refused and appealed responses per day; a sudden jump means a bad prompt change or a provider change. Verify: tests with a scripted model for each case: filter before text, filter mid-text, length, a normal refusal, and a moderated request. Assert on the message shown, the preserved text and the logged event.

Explain it without notes

How do a model refusal, a provider filter and your own moderation differ in what your code sees?

What should the UI do when a content filter stops an answer halfway?

What is the Scunthorpe problem and how do you avoid it?

Why should an appeal report not include the user's text by default?

Practice

In the first example, add a fifth case "error" that uses fail: { status: 500 } and show what the user sees. Which finish reason is there?

Add a rule to the second example: messages longer than 20 characters are allowed, but a message with a blocked word is allowed when it is wrapped in quotation marks. Is that a good rule?

Trade-offs

↔
Strict filters protect your brand and also block real customers. An appeal path and a review of appeals is how you find the right setting.
↔
Moderating before the model saves cost and avoids unsafe output, and adds latency to every request. Moderating replies after the fact catches more but happens after the user may have read it, unless you hold the stream.

Done when you can

I can tell a model refusal, a provider filter stop and my own moderation apart.
I can read finishReason and choose a state and message for each value.
I can show a partial answer that was stopped without hiding or faking it.
I can write whole-word, Unicode-aware moderation.
I can add an appeal that does not copy the user's text.

Command Palette

Words you'll meet

Step by step

01Read the finish reason, not only the text

02Pick the message for each case

03Moderate the request before the model

04Send the appeal without the text

Try it yourself

Match inside words

Handle the missing reason

Code & diagrams

Break it on purpose

Block by substring

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can