A gradient: calculus versus nudging
A tensor is a NumPy-like array that can run on a GPU and remember how it was computed, so PyTorch can find gradients automatically. Training a neural network is five repeated steps: forward pass, loss, gradients, update, repeat. You can run the whole idea with plain NumPy first.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write mse(y_true, y_pred) using NumPy.
Write gradient_step(w, grad, learning_rate) and use it three times to minimise w squared from w = 10.
Write linear_forward(batch, W, b) and check the output shape for a batch of 4 with 3 features and 2 outputs.
Explain it without notes
What are the five steps of a training loop?
What is a tensor, and how does it differ from a NumPy array?
What does autograd do, and why do we need it?
Your PyTorch model gives different outputs for the same input on repeated calls, and memory grows during evaluation. What are the likely causes and fixes?
Expected output
w=0.0: analytic -36.000 numeric -36.000
w=1.0: analytic -18.000 numeric -18.000
w=2.0: analytic 0.000 numeric 0.000
w=3.5: analytic 27.000 numeric 27.000