Advanced
How big is a LoRA adapter?
Write lora_params(d_in, d_out, r) returning the number of trained numbers for one matrix: two thin matrices of sizes d_in x r and r x d_out.
Write lora_percent(d_in, d_out, r) returning that number as a percentage of the full matrix (d_in * d_out), rounded to 3 decimals.
What Check tests
- A 4096 x 4096 matrix at rank 8 has 65,536 trained numbers
- A 4096 x 11008 matrix at rank 8 has 120,832
- Rank 8 is about 0.391 percent of a 4096 x 4096 matrix
- Rank 64 is 3.125 percent
Stuck? One hint at a time
The solution unlocks after your first Check. Try first: that is where the learning happens.
main.py
Sign in to run your code.
Output
Press Run to see what your program prints.