Topic 0.2
The Anatomy of an AI Feature
In one line
Every AI feature has three hops: the browser asks your server, your server asks the model provider with a secret key, and the answer travels back the same way. The key, the instructions and the checks live on your server, never in the browser.
Think of it like this
Ordering at a restaurant. You tell the waiter what you want. The waiter goes into the kitchen, which customers may not enter, and speaks to the chef. The food comes back through the waiter. You are the browser, the waiter is your server, the chef is the model provider. The waiter also checks your order makes sense before bothering the chef, and it is the restaurant, not you, that holds the kitchen key.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Route handler
- A function on your server that receives a request and returns a response.
/api/chatis handled by one. - API key
- A secret password for the provider's API. It identifies your account, and every call made with it is billed to you.
- Environment variable
- A setting stored on the server outside the code, used for secrets like API keys.
- Proxy
- A server that passes a request on to another server and returns the answer. Your route handler is a proxy to the model.
- System message
- Instructions for the model that your server adds and the user does not see.
- Role
- Who a chat message is from: system (your instructions), user (the person) or assistant (the model).
- 401 Unauthorized
- The status a server returns when the request has no valid key or login.
- 400 Bad Request
- The status a server returns when the request itself is wrong, such as an empty question.
Step by step
01The three hops
Follow one question through the system. The browser never talks to the provider, and the provider never sees the browser.
02What the browser sends
Only the question. No key, no model name, no instructions. If you open the browser's Network tab on a well-built AI app, this small body is all you find.
const res = await fetch("/api/chat", {
method: "POST",
body: JSON.stringify({ question: "Hello!" }),
});03What your server adds
The route handler reads the question, wraps it with the system message, attaches the key from an environment variable, and forwards it. This is the shape of a real handler; in the live example below the same code runs inside the page so you can watch it.
export async function POST(req: Request) {
const { question } = await req.json();
return fetch(PROVIDER_URL, {
method: "POST",
headers: { authorization: "Bearer " + process.env.PROVIDER_API_KEY },
body: JSON.stringify({
model: "provider/model-name",
messages: [
{ role: "system", content: "You are Mitra, a helpful assistant for the shop Chai & Co." },
{ role: "user", content: question },
],
}),
});
}04What happens without the server
The first live example ends by calling the provider straight from the browser. It has no key, so the provider answers 401. The only way to make that call work from the browser would be to ship the key to every visitor, which is the mistake this whole design exists to prevent.
05When the provider fails, the server answers for it
The provider can fail too: here it answers 429 (too many requests) with a message that names the organisation and the model. Your server logs that in full for you, then sends the browser a small answer of its own. The browser sees a calm sentence it can show; the internal details never leave the server. The status the browser receives is also your choice: 429 when the provider is busy, 502 (bad gateway: the server behind me failed) otherwise.
06The server is also the gatekeeper
The third example adds checks to the route: an empty question and a 501-character question are rejected with 400 and a clear message, and the model is never called for them. The counter at the end shows it: three questions asked, one model call paid for.
Browser-side checks (disabling the Send button on empty input) make the page pleasant. Server-side checks keep you safe, because anyone can send a request to your route without using your page at all.
Try it yourself
- 1
Get the key wrong
In the first example, change
const API_KEYto"sk-old-key". Predict what the browser prints forbrowser got:.It crashes on that line with
Uncaught TypeError: Cannot read properties of undefined (reading '0'), because the response is now a 401 with anerrorobject and nochoices. Real code must checkres.okbefore reading the answer: that is Topic 1.3. - 2
Let the browser choose the instructions
Imagine the browser sent the system message itself:
body: JSON.stringify({ system: "You are Mitra…", question }). Who could change it?Anyone, from the browser's developer tools, with no skill needed. They could make your assistant say anything under your shop's name, at your cost. Instructions belong on the server.
- 3
Tighten the limit
In the third example, change the limit from 500 to 20 characters and run it. Which line changes?
The first question (26 characters) is now refused too, and
model calls paid fordrops to 0. A limit is a product decision: too low and real questions are rejected.
Code & diagrams
All three parts run inside this one preview so you can read them together. In a real app the two serve() blocks are different machines: the first belongs to the provider, the second is your server. Only the last part is browser code.
Preview
Console
Nothing logged yet.Expected console output
your server got: "Hello!"
browser got: 200 "Namaste! I am Mitra. How can I help?"
direct call: 401 Missing or invalid API key.The first console line is what only your server knows. The second is all the browser gets: no organisation id, no model name. In a real app the first line goes to your logs, not the console.
Preview
Console
Nothing logged yet.Expected console output
(server log) provider said 429 Rate limit reached for org-abc123 on model-x
browser got: 429 {"error":"Mitra is busy. Please try again in a moment."}Three requests reach your server; only one reaches the model. The other two get a 400 with a message the page can show as it is.
Preview
Console
Nothing logged yet.Expected console output
Our return window is 7 days.
(400) Please type a question.
(400) That question is too long (500 characters at most).
model calls paid for: 1Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Send a body that isn't JSON
In the third example, replace the first console.log line with const raw = await fetch("/api/chat", { method: "POST", body: "hello" }); console.log(raw.status, await raw.json());
Myth vs fact
Myth
It's fine to put the API key in frontend code if the code is minified.
Fact
Minified code is still readable, and the key also appears in plain text in the Network tab. Any key sent to a browser is public.
Myth
The frontend can validate the question, so the server doesn't need to.
Fact
Requests can be sent to your route without your page. Only server-side checks actually protect cost and safety; browser checks are for comfort.
Myth
The server just forwards the request, so it's optional.
Fact
It holds the key, sets the instructions and model, enforces limits and checks who is asking. That is the control point of the whole feature.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Treat the proxy route as a public API of your product. It needs authentication, per-user rate limits and input-size caps, or someone will script it and spend your budget (Topics 1.5 and 13.2). The provider's own rate limit protects the provider, not your bill.
- ▸
A streamed response passes through every layer between the provider and the browser: your server, a load balancer (which spreads requests over several servers), a CDN (a network of caching servers close to users). Any layer that buffers the response turns your stream back into one big delayed blob. Symptoms: text arrives all at once in production but streams in development. Check for buffering and compression settings on each hop.
- ▸
Never forward the provider's raw error body to the browser. It can contain your account or organisation id, the model name and internal messages. Map failures to your own small set of error codes and messages (Topic 1.3).
Remember this
- 1
The problem this solves: The obvious way to add AI to a page is to call the model's API straight from the browser. That puts your secret key in code anyone can read, and within hours someone is using it at your expense. This topic shows the shape every real AI feature has instead, and what each part is responsible for.
- 2
Hop 1: browser to your server. The browser sends only what the user did: their question, perhaps the conversation so far. It sends it to an address on *your* site, such as
/api/chat. It needs no key, because it is talking to your own server, which already knows who the user is (cookies, a session). - 3
Hop 2: your server to the provider. A small piece of server code, usually called a route handler (a function that receives a request and returns a response), takes the question, adds what the browser must not control, and calls the provider's API with the API key. An API key is a secret password that identifies your account and is billed for every call. It lives in an environment variable on the server (a setting stored outside the code), never in files sent to the browser.
- 4
Hop 3: the provider runs the model and sends the answer back to your server, which passes it on to the browser. With streaming, the pieces flow through both hops as they are produced; your server is a pipe that forwards them. The reply has the shape you saw in Topic 0.1: the text, a finish reason and the
usagecounts. This hop is where the money is spent: the provider bills the account that owns the key for the tokens in and out. So every check your server can make *before* hop 3 saves real money. - 5
Why not skip your server? Three reasons. Secrets: anything in browser code can be read by any visitor (View Source, the Network tab), so a key there is public. Control: the server decides the model, the instructions and the limits; if the browser chose them, a visitor could change them. Checks: the server can refuse an empty or enormous question *before* paying for a model call, and can enforce that this user is allowed to ask at all.
- 6
The server usually adds a system message: instructions the user doesn't see, such as "You are Mitra, a helpful assistant for the shop Chai & Co. Answer only about orders, shipping and returns." A chat request is a list of messages, each with a role:
system(your instructions),user(what the person typed) andassistant(what the model replied earlier). Topic 1.1 goes through this format field by field. - 7
What the frontend owns in this picture: collecting the input, showing every state (waiting, streaming, done, failed, stopped), rendering the answer safely, and reporting problems in words a person understands. Where things go wrong: each hop adds delay and each can fail on its own. Your server can reject the request (400), the provider can refuse it (401 for a bad key, 429 when you call too often), the model can be slow, or the connection can drop midway. Phases 1 and 3 handle each one.
Explain it without notes
Describe the path of one question in an AI feature, hop by hop.
Why must the API key stay on the server?
What does the server add to a request that the browser must not control?
Why validate on the server if the page already validates?
Practice
Draw (in words) the three hops for a "summarise this article" button. What does each hop send?
Add a third check to the route in the third example: refuse any question that contains the word password, with status 400 and a helpful message. Why might a real product do something like this?
Trade-offs
- ↔
A proxy route adds one network hop and some server cost, and you must run and secure it. In return you keep the key secret and gain a place for validation, limits and logging. There is no safe alternative for a public web app.
- ↔
Strict input limits protect your budget but reject some real users. Start with generous limits, measure what people actually send, and tighten with data.
Done when you can
I can draw the three hops of an AI feature and say what each one sends.
I can explain why the API key lives in a server environment variable.
I can name three things the server adds or enforces that the browser must not control.
I can explain the difference between a 400 from my server and a 401 from the provider.
I can explain why browser-side validation alone is not enough.