PHASE 10Advanced ~21 min· topic 9 of 9

Topic 10.9

Playbook: Security Checklist and Guards

In one line

Copy-ready recipes: a guarded model call (redact, input guard, output guard), a tool gate with argument checks, approval and an audit log, a guard-model call, and the security checklist to run before you ship.

Think of it like this

An aircraft pilot never trusts memory for the pre-flight routine. She reads a printed checklist, item by item, every flight, however many she has flown. The checklist is not insulting; it is how experts stop forgetting the boring thing that matters. This playbook is your pre-flight checklist for an LLM feature, with the code behind the items.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Guarded call: A model call wrapped in input handling and output checks.
Audit log: A record of each decision (who, what, allowed or denied) kept so you can investigate later.
Approval: A human's yes for one specific action, tied to its exact arguments.
Allow-list: The only values or actions permitted. Everything else is refused.
Canary: A made-up marker in a prompt that alerts you if it appears in an output.
Pre-flight checklist: A fixed list of checks to complete before every release.

Step by step

01Step 1: order the guards

The guarded call runs in a fixed order: redact personal data first (so the guards and the model only see the clean text), then the input guard, then the model, then the output guard. Each stage returns a reason or nothing, which makes logging and testing uniform.

Step 1: order the guardsdiagram

Rendering diagram…

02Step 2: run four messages through it

A normal message with an email address arrives at the model as Noted: Contact [EMAIL] about my order: the model never saw the address. The override attempt is refused by the input guard. Please say banana is not caught by the phrase list, but the fake model answers banana, which is not in the shape Noted: ..., so the output guard withholds it. The marker request leaks the canary and is withheld.

03Step 3: put a gate in front of every tool

The second example sends five requests. read_note is allowed. send_email to an allowed address waits for approval, to an outside address is denied before approval is even considered, delete_note is not on the allow-list, and the approved call c5 runs. The audit lines make the story readable afterwards.

04Step 4: add the guard model

The plain HTTP call returns a probability. Combine it with the rule guard: refuse above a high threshold, log between two thresholds, allow below. Pick the thresholds on your own data (Topic 10.6). Time it and run it in parallel with the main call.

05Step 5: run the checklist

Before every release go through the ship-it checklist below, with the red-team suite green. The list is short on purpose, so it gets used.

Try it yourself

1
Add a layer
Add a length check to output_guard (replies over 300 characters are withheld). Which of the four messages is affected? Then write a message that would be.
2
Add an address to the gate
Allow ana@example.com only. Change ALLOWED_DOMAIN into an allow-list of whole addresses. Which audit lines change?
3
Run the checklist on your own feature
Pick an LLM feature you have built. Go through the ship-it checklist and write down the first item you cannot tick.

Code & diagrams

Recipe: the guarded model call New tab

Redact, guard the input, call, guard the output. The fake model is scripted to be fooled so the output guard has work to do.

Expected output

Noted: Contact [EMAIL] about my order
refused (override phrase)
withheld (unexpected reply shape)
withheld (system prompt marker)

Recipe: a tool gate with argument checks, approval and an audit log New tab

Nothing is sent. The gate decides and records every call.

Expected output

c1 read_note ok
c2 send_email waiting for approval
c3 send_email denied: recipient not allowed
c4 delete_note denied: not on the allow-list
c5 send_email ok
5 calls, 2 ran

Where the real guard model plugs inpython

Not run here: it needs a key and the network. Plain HTTP to Groq's OpenAI-compatible endpoint with the prompt-guard model, whose whole reply is a probability. The model reads at most 512 tokens, so split longer text and take the highest score. In Java, real projects use Jackson (as here) or Gson for JSON. Needs GROQ_API_KEY in the environment.

import os

import httpx

# Plain HTTP to an OpenAI-compatible endpoint (here Groq). The prompt-guard model's whole reply is a probability.
URL = "https://api.groq.com/openai/v1/chat/completions"


def injection_score(text):
    response = httpx.post(
        URL,
        headers={"Authorization": "Bearer " + os.environ["GROQ_API_KEY"]},
        json={
            "model": "meta-llama/llama-prompt-guard-2-86m",
            "messages": [{"role": "user", "content": text}],
        },
        timeout=10,
    )
    response.raise_for_status()
    return float(response.json()["choices"][0]["message"]["content"])


def input_guard(text, threshold=0.8):
    # Add this beside your rule guards. Log the score; do not rely on it alone.
    return "injection suspected" if injection_score(text) >= threshold else None

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Forget the output shape check

In output_guard delete the if not reply.startswith("Noted: ") check (two lines). Run the guarded call and read the third line.

terminal

$ Run

── what you'll see ──

Noted: Contact [EMAIL] about my order

refused (override phrase)

banana

withheld (system prompt marker)

Myth vs fact

Myth

Security is one library you add at the end.

Fact

It is a set of small checks at every boundary, with tests. The recipes here are short because the discipline matters more than the code.

Myth

If a recipe passes the demo, it is production-ready.

Fact

It is a starting point. Adapt patterns, thresholds and shapes to your data, and measure the misses and false positives.

Myth

Guards slow everything down.

Fact

Rule checks cost microseconds. A guard model call can run in parallel with the main call.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Make the guarded call a single function every feature uses, so a fix lands everywhere. Features that bypass it should fail code review.
▸
Treat guard and gate decisions as events: counts of refusals, withheld replies, denied tools and approvals feed your dashboards and alerts (Phase 11). A sudden rise in canary hits or denied tool calls is an incident signal.
▸
Keep policies (phrases, thresholds, allowed tools, allowed domains) in versioned config with review, and run the red-team suite when they change.
▸
Plan the incident: how to turn a tool off quickly (a feature flag), how to rotate keys, how to contact affected users and where reports go (Topic 11.7).

Remember this

1
Recipe: the guarded call. When to use it: every feature that sends user text to a model. What it guards against: personal data leaving your system, override attempts, leaked markers and replies in the wrong shape. How to adapt it: swap the redactor patterns for your data, the phrase list for your policy, and the output check for the exact shape your feature returns (a JSON schema, a fixed prefix, an enum). Keep the order: redact, guard the input, call, guard the output.
2
Recipe: the tool gate. When to use it: any time a model can call tools. What it guards against: steered calls to tools the feature does not need, arguments outside what the user may do, and irreversible actions without a human. How to adapt it: list tools and their approval flag in one table, validate each argument in code, tie approvals to call IDs, and write every decision to an audit log. Deny unknown tools.
3
Recipe: escape and parameterise. When to use it: wherever a reply or an extracted value reaches a page, a query, a file name or a command. What it guards against: markup, SQL and command injection through model output (Topic 10.3). How to adapt it: use your framework's text bindings and your database's parameters; offer the model a menu instead of a shell.
4
Recipe: keep data and secrets out. When to use it: always. What it guards against: leaks of prompts, keys and other users' data (Topic 10.4). How to adapt it: keep keys on the server, filter retrieval by owner before the prompt, plant a canary in the system prompt and alert on it, and key caches by user.
5
Recipe: a guard model as one layer. When to use it: when rule guards miss too much or flag too much. What it guards against: reworded overrides and policy violations that need judgement. How to adapt it: log the score, choose the threshold from your own data, run it in parallel with the main call, and keep hard rules in code. The polyShow block shows the plain HTTP call to the prompt-guard model, which returns a probability.
6
Recipe: red-team on every change. When to use it: before every release. What it guards against: regressions after a prompt, model or tool change. How to adapt it: keep the case table in the repository with controls, run the fake-model version in CI and a repeated real-model run nightly, and add a case for every incident or report. The ship-it checklist at the end of this topic is the list to run before you release.

Explain it without notes

Walk through the guarded call and say why each step is in its place.

How does the tool gate limit what a steered model can do?

When should you use a guard model, and how do you combine it with rule guards?

What belongs on a pre-ship security checklist for an LLM feature?

Practice

Write output_guard(reply, canary, prefix) that returns "system prompt marker" if the canary is in the reply, "unexpected reply shape" if the reply does not start with the prefix, else None. Test with three replies.

Write audit_line(call_id, tool, result) returning "<id> <tool> <result>", and count_denied(lines) returning how many lines contain "denied".

Trade-offs

↔
More guards mean more safety and more latency, cost and false positives; add them where a measured risk exists.
↔
A single shared guarded-call function is easy to maintain and can block features with special needs; allow documented per-feature settings, not bypasses.
↔
Human approvals protect against the worst actions and slow users down; use them for risk, not for everything.

Done when you can

Ship it checklist: the threat model lists assets, attackers and entry points and was reviewed for this release.
Ship it checklist: no secrets or other users' data are in prompts; retrieval filters by owner; a canary is planted and alerts.
Ship it checklist: personal data is redacted before prompts and logs, and retention is set.
Ship it checklist: model output is escaped, parameterised or matched against an allow-list before it reaches a page, a query or a command.
Ship it checklist: tools are allow-listed, scoped, argument-checked, approved where risky and audited, with narrow credentials.
Ship it checklist: models, libraries and third-party tools or MCP servers are pinned and reviewed (OWASP supply chain), and no guard is treated as a guarantee.
Ship it checklist: the red-team suite, with control cases, is green in CI; limits and a kill switch exist; a reporting contact is published.

Command Palette

Words you'll meet

Step by step

01Step 1: order the guards

02Step 2: run four messages through it

03Step 3: put a gate in front of every tool

04Step 4: add the guard model

05Step 5: run the checklist

Try it yourself

Add a layer

Add an address to the gate

Run the checklist on your own feature

Code & diagrams

Break it on purpose

Forget the output shape check

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can