Command Palette

Search for a command to run...

Hectal
Phase 13Advanced14 of 16 in AI Frontend

Cost, Limits and Production

Show what an answer costs, stop one user from emptying the budget, keep working when a provider is down, cache safely, measure without spying, and test every state.

A demo works once. A product works on a bad day: the provider is slow, one user asks a thousand questions, the bill arrives, and nobody can tell from the logs whether the feature is any good. This phase is about everything that happens after "it streams on my laptop".

You will count tokens before and after a request, enforce daily limits on the server and show them honestly in the screen, fall back to a second model and a kill switch, cache answers without leaking one person's data to another, measure speed and failures without logging what people typed, and test every state of the chat with a scripted model.

Every example runs against the scripted stand-in model, with made-up prices and limits. No real price, model name or rate limit appears anywhere in this phase: they change often, so read your provider's current price and limits pages. By the end Mitra has a usage meter, a daily limit, a backup model, a cache, a kill switch and a test suite.

0/8 · 0%
8 topics ~4 h 19 code blocks & diagrams
Start with the first topic
1
13.1

Tokens and Cost in the UI

A model charges for tokens in and tokens out, and every request resends the whole conversation. The screen can estimate the cost before sending, show the real usage after, warn about long pastes, and say when an answer was cut off by a length limit.

30 min 3 code practice

2
13.2

Quotas and Budgets

A quota stops one user from spending the whole budget. It is counted and enforced on the server, before the model is called, and the screen shows how much is left, warns near the limit and explains the wait at the limit with a 429 and a retry-after header.

30 min 2 code practice

3
13.3

Model Fallbacks and Degraded Modes

Providers have bad hours. A second model can answer when the first one fails in a way another model could fix, the screen must say which mode it is in, and a kill switch lets you turn the feature off in seconds with a useful page left behind.

32 min 3 code practice

4
13.4

Caching AI Responses

A cache can answer the same general question for free. It must key on a normalised question and the facts it depends on, expire or be replaced when facts change, replay in the same streaming format, and never hold personal or incomplete answers.

28 min 2 code practice

5
13.5

Measuring an AI Feature

You cannot improve what you cannot see. Measure time to first token, speed, errors, stops and feedback as counts and buckets, and keep the text people typed and the answers they got out of your logs.

30 min 3 code practice

6
13.6

Testing AI UI

Test the screen, not the model: replace the model with a script, drive the component like a user, wait for what appears on screen instead of for timers, and cover every state (waiting, streaming, done, 429, broken stream, stopped).

29 min 2 code practice

7
13.7

Evals a Frontend Team Owns

An eval is a saved set of cases with automatic checks. A frontend team owns the UI-critical ones: does the reply parse, fit, avoid raw HTML, mark refusals. They run on scripted replies in every build and on real replies when the prompt or model changes.

26 min 2 code practice

8
13.8

Build: Mitra in Production

Mitra gets a usage meter, a daily limit enforced on the server, a backup model, a cache for general questions, a kill switch, content-free metrics and a test suite. It can now run on a bad day without surprising the user or the bill.

38 min 2 code practice

Practise this phase

Hands-on exercises: write the code, see your page live, and press Check to have it tested automatically.

1

Count Before Sending

The box #prompt is a textarea. Show the estimated size in #count as N of 20 tokens, using the estimateTokens function that is already written. Show the count at the start too (0 of 20 tokens).

Beginner
2

Show the Wait

The route /api/chat answers { text }, except for a question that contains the word limit: then it answers 429 with a retry-after header of 90 seconds. Clicking #send posts the text of #prompt as { question } and shows text in #answer. That part works.

Intermediate
3

Backup Mode and the Kill Switch

The route answers three requests in turn. The 1st is { text, mode: "main" }. The 2nd is { text, mode: "backup" }. From the 3rd on, the kill switch is on: status 503 with { error: "ai_paused" }.

Intermediate
4

Normalise the Question

The page keeps a small cache of answers in a Map. Clicking #ask reads #prompt, makes a key with normalise(question), and either shows the saved answer (#source says cache) or asks the route and saves it (#source says model). #answer shows the text, which ends with the model's call number.

Intermediate
5

Log Only What Is Allowed

Each click on #fire sends the next event to your track(props) function and prints what you decided to keep on a new line of #logged as JSON. ALLOWED lists the keys and the values each may have.

Advanced
6

Build the Usage Meter

The route allows 3 questions. Every answer carries the headers x-quota-used and x-quota-limit. Clicking #ask shows the answer in #answer and updates #meter to N of 3 questions left today (N = limit - used).

Advanced