The naive pipeline: a document steers the app
The planted lines are harmless stand-ins. The fake model obeys them, like a fooled real model would.
Prompt injection is text that makes a model follow someone else's instructions. No single trick stops it, so you defend in layers: keep data apart from instructions, check the output, give tools the least power, and add a detector.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write safe_reply(reply, allowed_tools) that returns the reply when it starts with Summary: , returns blocked tool: <name> when the reply is CALL <name> for a tool not in allowed_tools, and returns blocked: not a summary for anything else.
Write wrap_document(doc) that returns the document inside <document> and </document> tags, each on its own line, and removes any </document> already inside the text so the data cannot close the tag early.
Explain it without notes
What is prompt injection and why can't it be fixed once and for all?
What is the difference between direct and indirect prompt injection, and which is harder to defend?
Why are delimiters not a defence on their own, and what are they good for?
Describe a layered defence for a feature that summarises untrusted documents.
How would you test that your defences work without writing a real attack?
Expected output
clean -> Summary: sales rose 8 percent and costs fell 2 percent.
banana -> banana
tool -> RAN TOOL send_email (pretend)