Command Palette

Search for a command to run...

Hectal
Phase 1Beginner2 of 15 in LLM Engineering

How LLMs Work (for Engineers)

Tokens, embeddings, attention, the context window, sampling, training, reasoning and hallucination, each built by hand with small numbers.

You do not need to train a model to build with one, but you do need a working picture of what happens inside the box. This phase builds that picture from small pieces you can run: a tokenizer that merges letters into pieces, three-number embeddings and cosine similarity, attention weights on three tokens, a context window that overflows, and a sampler with a temperature knob.

Each mechanism is explained by the engineering question it answers. Why is the bill counted in tokens? Why can a model not count letters? Why does the same prompt give different answers? Why does a long chat forget its beginning? Why does a model say something false in a confident voice? Knowing the cause tells you which fix to reach for in later phases.

Every example prints the same output in Python, Java and JavaScript, and the real-model labs let you see temperature and reasoning tokens on a real hosted model. The phase ends with a cheat sheet of recipes: a token budget helper, temperature defaults per task and a hallucination-risk checklist.

0/10 · 0%
10 topics ~3 h 16 code blocks & diagrams
Start with the first topic
1
1.1

Tokens

A token is the small piece of text a model reads and writes: often a whole common word, part of a rare word, a digit group or a symbol. Models work on token ids, not letters, which explains the bill, the limits and why spelling questions are hard.

22 min 2 code practice

2
1.2

Counting Tokens

Tokens decide your cost, your limits and your speed, so you need to count them. The 4-characters-per-token rule is a fine first guess and a bad final answer: measure with the model's own tokenizer or read the usage field of the reply.

18 min 1 code practice

3
1.3

Embeddings: Meaning as Numbers

An embedding turns text into a list of numbers so that similar meanings get nearby lists. Cosine similarity measures how close two lists point, which is the base of semantic search and RAG.

18 min 1 code practice

4
1.4

Attention in Plain Words

Attention lets every token look at the other tokens and decide which ones matter for its meaning. Scores become weights with softmax, and the token's new vector is a weighted average of what the others offer. Stacking this many times is the transformer.

19 min 1 code practice

5
1.5

The Context Window

The context window is the fixed number of tokens a model can handle in one call, input and output together. The API remembers nothing between calls, so your app re-sends the history each time and must trim it when it grows too long.

19 min 1 code practice

6
1.6

Sampling: Temperature, Top-k and Top-p

A model produces a score for every possible next token. Softmax turns scores into probabilities, and a sampler picks one. Temperature, top-k, top-p and the seed control how adventurous that pick is, which is why the same prompt can give different answers.

20 min 3 code practice

7
1.7

How Models Are Made

A chat model is built in stages: pretraining on huge amounts of text to predict the next token, instruction tuning on examples of helpful answers, and preference tuning on human judgements. Each stage explains a behaviour you will meet, from knowledge cutoffs to refusals and flattery.

19 min 1 code practice

8
1.8

Reasoning Models

Reasoning models write hidden thinking tokens before the visible answer. That helps on multi-step problems, and it costs time and money, and the thinking shares your max_tokens budget, which creates a classic trap: an empty reply with finish_reason: length.

19 min 2 code practice

9
1.9

Hallucination from First Principles

A model predicts likely text, not true facts, so it can produce fluent, confident answers that are false. Knowing the causes tells you the defences: ground answers in sources, ask for evidence, allow I do not know, and verify what matters.

19 min 1 code practice

10
1.10

Playbook: Model Behaviour Cheat Sheet

Four recipes to keep next to every LLM feature: an input-budget helper, a daily-cost projection, starting temperatures per task, and a hallucination-risk checklist. Copy them, then replace the guesses with your own measurements.

20 min 3 code practice

Practise this phase

Hands-on exercises: write the code, run it, and press Check to have it tested automatically.

1

Count neighbouring pairs (the heart of BPE)

Write pair_counts(words) where words is a dictionary of word to how often it appears, e.g. {"low": 5, "lower": 2}.

Beginner
2

Cosine similarity

Write cosine(a, b) for two lists of numbers of the same length: the dot product divided by the product of the two lengths (square roots of the sums of squares).

Beginner
3

Softmax with temperature

Write softmax_t(logits, t) that returns a list of probabilities: divide each logit by t, subtract the largest value for stability, apply exp, and divide by the total.

Beginner
4

Top-p (nucleus) filtering

Write top_p(probs, p) where probs maps tokens to probabilities. Sort tokens by probability, highest first (ties by token name). Keep tokens in that order until their running total is at least p, including the token that reaches it.

Intermediate
5

Trim a chat to fit the context window

Write trim_history(messages, budget, count). messages is a list of (role, text) pairs, count(text) returns a token count and budget is the most input tokens allowed.

Intermediate