Topic 1.10
Playbook: Model Behaviour Cheat Sheet
In one line
Four recipes to keep next to every LLM feature: an input-budget helper, a daily-cost projection, starting temperatures per task, and a hallucination-risk checklist. Copy them, then replace the guesses with your own measurements.
Think of it like this
A car's dashboard. You do not need to understand the engine to drive safely, but you need a fuel gauge, a speed reading and warning lights. These recipes are the dashboard for a model feature: how much fits, what it costs, how adventurous it is and how risky an answer is.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Input budget
- The number of input tokens you may send once the reply and a safety margin are reserved.
- Safety margin
- Extra room left unused to cover counting errors.
- Cost projection
- An estimate of spending from call volume, token counts and prices.
- Risk score
- A number from weighted yes/no questions that tells how much protection a feature needs.
- Pinned version
- A fixed, dated model name used instead of an alias that can change.
Step by step
01Step 1: size the input
Start every feature by writing down the window of your model, the longest reply you will allow and a margin. The budget recipe turns them into one number: 6,600 for an 8,000-token window with a 1,000-token reply and a 5 percent margin.
02Step 2: put money on it
The cost recipe multiplies volume by tokens by price. 50,000 calls a day at 1,500 input and 300 output tokens is 60 dollars a day at the example prices, 1,800 a month. Short answers cut it to about 49 a day. Seeing this early decides model choice and prompt length.
03Step 3: pick the temperature on purpose
The table in the second example gives starting points. Write the chosen value into the request explicitly instead of relying on the default, and test two neighbouring values on a few real inputs.
04Step 4: score the risk
The FAQ bot must answer from documents, users can ask anything, and a wrong answer causes harm: score 7, high. A poem generator scores 1, low. A docs helper scores 4, medium. The level decides the protections: from basic checks up to quote verification and human review.
05Step 5: plug in the real call
The last block shows a real call with temperature, max_tokens and the fields to log. The recipes decide the numbers you put into it, and the reply's usage and finish_reason feed back into the recipes.
Try it yourself
- 1
Your own feature
Pick a feature you might build. Fill in the window of a model, a reply limit, calls per day and average token counts. Run the budget and cost recipes and write down the daily cost.
- 2
Add a question
Add the risk question
answer is shown without human reviewwith weight 2, in all three languages. Does any scenario change level? - 3
Test temperatures
Open the temperature lab in Topic 1.6 and compare temperature 0 and 1.5 on a task from your own list. Does the recipe's starting value match what you prefer?
Code & diagrams
Prices are example numbers. Keep them in configuration and check your provider's price page.
Expected output
input budget, 8000 window, 1000 reply: 6600
input budget, 128000 window, 4000 reply: 117600
chatty answers: 60.00 dollars a day, 1800.00 a month
short answers: 48.75 dollars a day, 1462.50 a monthStarting points, not laws. Test them on your own examples.
Expected output
extract fields temperature 0.0
classify temperature 0.0
write code temperature 0.2
summarise temperature 0.3
chat temperature 0.7
brainstorm temperature 1.0
FAQ bot: score 7, risk high -> ground, verify quotes, add human review
poem generator: score 1, risk low -> ship with basic checks
docs helper: score 4, risk medium -> ground with sources and allow I do not knowNot run here: it needs your own GROQ_API_KEY and the network. Groq is used because its API is OpenAI-compatible and has a free tier; other providers change only the base URL, the key and the model name.
# pip install openai (Groq speaks the OpenAI-compatible API, so the same SDK works with a different base URL)
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ["GROQ_API_KEY"], base_url="https://api.groq.com/openai/v1")
resp = client.chat.completions.create(
model="openai/gpt-oss-20b",
messages=[{"role": "user", "content": "Classify the sentiment of: I love this phone."}],
temperature=0.0, # extraction and classification: as repeatable as possible
max_tokens=400, # room for hidden reasoning tokens plus the answer (Groq's newer name for this cap is max_completion_tokens)
)
choice = resp.choices[0]
print(choice.message.content)
print("finish_reason:", choice.finish_reason) # "length" means the reply was cut off
print("tokens:", resp.usage.prompt_tokens, resp.usage.completion_tokens)
Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Forget the reply reserve
In the Python budget recipe change the return line to return window - margin (leave out max_output).
Myth vs fact
Myth
Defaults are fine, I do not need to set temperature or max_tokens.
Fact
Defaults vary by provider and model and can change. Set both explicitly for every call.
Myth
Cost is small, so I can ignore it until later.
Fact
Cost scales with volume and prompt length. A quick projection before launch avoids a bad surprise.
Myth
A risk checklist is paperwork.
Fact
It makes you decide, before launch, how much checking a feature needs. Skipping it moves the decision to production.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Log the inputs to these recipes in production: window used, tokens in and out, finish_reason, model version, temperature. The first alarm is usually a creeping average of input tokens.
- ▸
Keep these numbers in one configuration object per feature (model, temperature, max_tokens, margin, prices). Changing a model then touches one place, and a test can assert that every feature sets them.
- ▸
Re-run the cost projection when you add retrieval, tools or a reasoning model. Each multiplies tokens per call in ways that are easy to forget.
Remember this
- 1
Recipe: input budget. When to use it: before any call that includes history or documents. It computes
window - max_output - safety margin, the number of input tokens you may send. It guards against the overflow error and against replies cut short because the input used the window. Adapt it by setting the margin to the error of your token counter (5 to 10 percent when you estimate, less when you use the real tokenizer). - 2
Recipe: daily cost projection. When to use it: in the design review, before launch. It multiplies calls per day by average input and output tokens and your price per million tokens. It guards against the surprise bill: the example shows how shortening answers from 300 to 150 tokens cuts the daily cost by about a fifth. Keep prices in configuration, and re-run with real averages from your
usagelogs. - 3
Recipe: temperature defaults. When to use it: when you start a new feature. 0.0 for extraction and classification, 0.2 for code, 0.3 for summaries, 0.7 for chat and 1.0 for brainstorming. It guards against random output where you need consistency and against dull output where you want variety. Adapt it by testing a handful of values on your own examples (Phase 9); model families differ.
- 4
Recipe: hallucination-risk checklist. When to use it: when deciding how much protection a feature needs. Four yes/no questions with weights (answers must come from your documents, asks about recent events, users can ask anything, a wrong answer causes harm) give a score and an action level. It guards against shipping a high-risk feature with low-risk protections. Adapt it by changing the questions and weights for your domain and writing down what each level requires.
- 5
Before-you-ship list for this phase. Token counts come from the right tokenizer or from
usage. Input plus reply fits the window with a margin.finish_reasonis checked on every reply, includinglengthwith empty content. Temperature is chosen on purpose. Answers that must be true are grounded and verified. The model version is pinned.
Explain it without notes
How do you compute how many input tokens you can send?
How do you estimate the monthly cost of a feature before building it?
How would you choose a starting temperature for a new feature?
How do you decide how much hallucination protection a feature needs?
Practice
Write max_calls_per_day(budget_dollars, tokens_in, tokens_out, price_in, price_out) returning the whole number of calls a daily dollar budget pays for (prices per million tokens). Test: budget 60, 1500 in, 300 out, prices 0.50 and 1.50.
Write risk_level(score) returning low for 0 to 1, medium for 2 to 4 and high for 5 and above. Print the levels for scores 0, 3 and 7.
Trade-offs
- ↔
Large safety margins avoid overflow and waste window space.
- ↔
Low temperature gives consistency and less variety, so choose by task.
- ↔
Heavy hallucination protection costs latency and tokens, so match it to the risk level.
Done when you can
Before you ship: input budget is computed from window, max reply and margin.
Before you ship: daily and monthly cost are projected with real averages.
Before you ship: temperature and max_tokens are set explicitly and chosen on purpose.
Before you ship: finish_reason and empty content are checked on every reply.
Before you ship: risk level is scored and the matching protections are in place.
Before you ship: the model version is pinned and the usage is logged.