Command Palette

Search for a command to run...

Hectal
Phase 9Advanced10 of 16 in AI Frontend

Prompt Injection and Safe Frontends

Defending an AI feature: untrusted text, prompt injection, unsafe output, data leaks through images and links, least privilege, refusals and privacy.

An AI feature reads text from many places: the user, documents they paste, web pages, database rows, other tools. The model cannot tell which of that text is an instruction from you and which is only data. Anyone who can put words in front of the model can try to give it orders. This phase is about building screens and routes that stay safe when that happens.

Everything here is defensive. For each threat you learn what it is, how to detect it, how to defend against it and how to verify the defence works. The attacks in the examples use harmless stand-ins (a document that says to reply with the word BANANA, an image pointing at a made-up attacker.example address). The scripted model plays a hostile or gullible model, and your code handles it safely: the lesson is always the defence.

You will treat model output like user input, decide which links and images may appear, give each tool only the power one user needs, show a refusal respectfully, and keep personal data out of places it does not belong. The phase ends by hardening Mitra and checking it against a short list you can run yourself.

0/8 · 0%
8 topics ~3 h 16 code blocks & diagrams
Start with the first topic
1
9.1

The Threat Model for an AI UI

A threat model lists what an attacker controls, what they want and what the worst outcome is. For an AI feature, the attacker controls every piece of text the model reads that you did not write, and the model cannot tell that text from your instructions.

23 min 2 code practice

2
9.2

Prompt Injection, Direct and Indirect

Prompt injection is text that tries to give the model new orders. It can be typed by the user (direct) or hidden in a document, page or tool result the model reads (indirect). A frontend cannot make a model immune, so it contains the damage and shows the user what happened.

24 min 2 code practice

3
9.3

Insecure Output Handling

A model's reply is untrusted input: it may repeat hostile text it read. Render it as text, allow only safe link protocols, never run generated code, and put any generated HTML in a sandboxed frame. Then write a test that proves no active content gets through.

24 min 2 code practice

4
9.4

Data Leaks Through the UI

Exfiltration means data leaving through a channel nobody meant to open. In chat UIs the classic channel is a Markdown image: the browser fetches it automatically, so an address that contains the user's data is sent without a click. Allow-list hosts, proxy or block images, and log blocks without copying the data.

24 min 2 code practice

5
9.5

Least Privilege for AI Features

Least privilege means each part of the system gets only the power it needs. For an AI feature that means: tools are built per request for the signed-in user, the server (not the prompt) decides who may do what, risky actions need approval, and the model's arguments are never trusted.

23 min 2 code practice

6
9.6

Refusals, Filters and Moderation in the UI

Sometimes an answer is stopped on purpose: the model declines, a provider filter cuts it off (finish reason content-filter), or your own moderation blocks the request. Each needs its own calm, honest message, a way to ask for a second look, and care for a half-written answer.

23 min 2 code practice

7
9.7

Privacy and Personal Data

Whatever the user types may leave the browser, reach your server, a model provider and your logs. Decide what is sent, remove personal data that is not needed before it leaves, ask for consent, keep only what you must for as short a time as you can, and never write prompts to logs carelessly.

23 min 2 code practice

8
9.8

Build: Mitra Under Attack

Mitra gets its defences: a server that builds the system message and the tools for the signed-in user, a screen that renders replies as text with checked links and no outside images, and a short checklist you run to prove each defence works.

27 min 2 code practice

Practise this phase

Hands-on exercises: write the code, see your page live, and press Check to have it tested automatically.

1

Allow Only Safe Links

A reply contains four links. Write safeHref(raw) so it returns the full address for https: and mailto: links and null for everything else. Parse with new URL(raw, "https://chai.example"), and use try/catch for text that is not a URL.

Intermediate
2

Block the Quiet Leak

The reply below contains four image addresses. Write imageAllowed(url) so it returns true only when the address uses https:, has no user name or password, and the host is exactly chai.example or cdn.chai.example.

Intermediate
3

Only the Signed-In User's Orders

The route /api/order returns any order for any id. The signed-in user is asha (the constant SESSION_USER). Change the route so it returns the order only when its owner is the session user, and { found: false } otherwise, with status 200 in both cases.

Advanced
4

Hide Personal Data Before Sending

The route /api/chat echoes what it received. When #send is clicked, replace every email address in the text of #prompt with [email] before posting it, show what the route received in #sent, and show 1 email hidden (or N emails hidden) in #note.

Advanced