Top-k, top-p and a seeded draw
The generator is the one used throughout this course, identical in all three languages.
A model produces a score for every possible next token. Softmax turns scores into probabilities, and a sampler picks one. Temperature, top-k, top-p and the seed control how adventurous that pick is, which is why the same prompt can give different answers.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write softmax_t(logits, t) and print the probabilities of [2.0, 1.0, 0.0] at T=1 with 4 decimals.
Write top_k_tokens(probs, k) where probs is a dictionary of token to probability, returning the k most likely tokens as a list (ties broken by name). Test it with {"a": 0.5, "b": 0.3, "c": 0.2} and k=2.
Explain it without notes
Explain the path from logits to a chosen token.
What does temperature do, mathematically and in practice?
How do top-k and top-p differ, and why use them?
Why does the same prompt give different answers, and how do you make behaviour more repeatable?
Expected output
top-k 3: Paris=0.7361 London=0.1643 Rome=0.0996
top-p 0.9: Paris=0.7361 London=0.1643 Rome=0.0996
8 draws: Paris Paris Paris Paris Paris Rome Paris Paris