Overfitting: train error falls, test error rises
Exact numbers depend on the seed. The pattern is what matters: degree 8 fits training points almost perfectly and does worse on new ones.
Machine learning fits a rule to example data. You split the data into train and test sets, fit a model on the training part, predict and score on the test part, and watch for overfitting. You can build a nearest-neighbour classifier and a polynomial fit from scratch with NumPy before using scikit-learn.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write train_test_split_idx(n, test_fraction, seed) returning shuffled train and test index lists.
Write accuracy(y_true, y_pred) with a length check and 0.0 for empty input.
Write predict_1nn(X_train, y_train, point) that returns the label of the nearest training row.
Explain it without notes
Why do we split data into training and test sets?
What is overfitting, and how would you detect and reduce it?
What is data leakage? Give an example involving scaling.
A fraud model reports 99 percent accuracy. Is it good? What would you look at instead, and how would you evaluate it properly?
Expected output
degree 1: train error 0.1059 test error 0.0111
degree 3: train error 0.0633 test error 0.0304
degree 8: train error 0.0011 test error 0.1971