Sampling in 30 lines: temperature and top-p
Four raw scores, softmax at three temperatures, and the top-p cut. The same numbers back the sampling answer below.
A dozen concept questions come up in almost every LLM engineering interview: tokens, context, sampling, hallucination, RAG against fine-tuning, evaluation, injection, structured output, cost and failures. This topic gives a model answer to each and points to the topic that taught it.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write request_cost(tokens_in, tokens_out, price_in_per_mtok, price_out_per_mtok) that returns the cost in dollars, with prices per million tokens. Print the cost of one request with 2,000 input and 300 output tokens at example prices of 0.50 and 1.50, and of 100,000 such requests.
Write top_k(probs, k) that keeps the k most likely tokens of a dictionary of probabilities and renormalises them so they add up to 1. Show it on four tokens with k = 2.
Explain it without notes
1. What is a token, and why does it matter for cost, speed and limits?
2. What is the context window, and what do you do when a conversation outgrows it?
3. Why does the same prompt give different answers, and what do temperature, top-k and top-p do?
4. Why do models hallucinate, and how do you reduce it?
5. When would you use prompting, RAG or fine-tuning?
6. Walk through a RAG pipeline and say where it usually fails.
7. How do you evaluate an LLM feature before and after launch?
8. What is prompt injection, and how do you defend against it?
9. How do you get reliable structured output from a model?
10. How do you reduce the cost and latency of an LLM feature?
11. How do you handle rate limits, timeouts and provider outages?
12. How does tool calling work, and what makes it dangerous?
Expected output
temperature 0.2: cat=0.993 dog=0.007 bird=0.001 fish=0.000
temperature 1.0: cat=0.609 dog=0.224 bird=0.136 fish=0.030
temperature 2.0: cat=0.434 dog=0.263 bird=0.205 fish=0.097
top-p 0.9 at temperature 1.0 keeps cat, dog, bird (together 0.970)