Preparing data: remove duplicates, split, check for leakage
Twelve tiny examples with two near-duplicates. The shuffle uses the course's seeded generator so every language agrees.
Fine-tuning continues a model's training on your own examples. It is good at teaching a format, a style or a narrow task, and poor at adding facts, which belong in retrieval. Try prompts and retrieval first.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write lora_params(d_in, d_out, r) and full_params(d_in, d_out), and print both for a 4096 x 4096 matrix with r = 8.
Write train_size(n, percent) that returns how many of n examples go to training using whole-number maths (n * percent // 100), and print it for 10 examples at 80 percent and for 7 examples at 80 percent.
Explain it without notes
When should you fine-tune rather than use retrieval?
Explain LoRA and why its adapter is small.
Why must the validation set be kept out of training, and how do you check?
In what order would you try to fix a feature that gives answers in the wrong format?
Expected output
kept 10 of 12; dropped duplicate ids: 3, 9
train ids: 8, 7, 4, 11, 6, 2, 10, 12
validation ids: 1, 5
validation prompts that also appear in train: 0
train labels: billing 3, bug 3, other 2
validation labels: billing 1, bug 1, other 0