Command Palette

Search for a command to run...

Hectal
Back to the lesson: Topic 10.6 — Moderation and Guard Models
LLM Engineering · Example 1 of 1

An input guard, an output guard and a chain

The fake model obeys a request for 'banana'. The cases include one miss and one false positive on purpose.

Guards check what goes into and comes out of a model. Simple rule guards are fast and predictable; guard models read meaning and follow a written policy. Both make mistakes in both directions, so you measure them and keep them as one layer among several.

Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.

Ready for more? This phase has 5 hands-on exercises with automatic checks.

Practice questions

Write the code in the editor, run it, then open the model answer to compare.

01

Write input_guard(text, phrases, max_chars) returning (True, "ok") or (False, reason). Test with a long text, a text containing a phrase in different capitalisation, and a normal text.

02

Write block_rate(guard, messages) that returns the fraction of messages for which guard(message) returns False, formatted to 2 decimals as a string.

Explain it without notes

01

What are input and output guards, and why do you need both?

02

What are the strengths and weaknesses of rule guards compared with guard models?

03

How would you choose a threshold for a guard that returns a score?

04

How do you reduce harm from false positives?

An input guard, an output guard and a chain
Sign in to run this example in your browser.

Expected output

normal -> Happy to help with: What are your opening hours?
override -> BLOCKED IN (override phrase: ignore the rules)
rephrased override -> banana
leaky reply -> BLOCKED OUT (banned word: password)
honest question -> BLOCKED IN (override phrase: ignore the rules)
too long -> BLOCKED IN (too long)
4 of 6 blocked