When "NA" is a country: how read_csv decides what is missing
By default read_csv treats several words as missing, including NA, N/A, null and an empty field. Namibia's code is NA, so the first load silently loses a country. Tell pandas exactly which tokens mean missing for which column. The last lines show that pandas skips NaN in sum and mean while NumPy does not, and what the nullable Int64 type does with a gap.
Real data arrives messy: missing values, duplicates, stray spaces, text where numbers should be and dates stored as strings. read_csv loads it, and a handful of pandas methods clean it before analysis or model training.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Load a CSV text with columns item,price where one price is missing, and fill the missing price with the median.
Clean the text column [" PEN ", "Book", "pen"] so all three entries match, and count the unique values.
Convert the strings ["2024-03-01", "oops"] to dates with coerce and count the missing results.
Load the CSV text code,n / NA,1 / IN,2 so that the code NA stays a real string, and print the list of codes.
Explain it without notes
How do you find missing values in a DataFrame, and what are your options for handling them?
Why can an integer column show as floats after read_csv?
What does errors="coerce" do in to_datetime? When is it dangerous?
Why does removing duplicates matter before training a model?
A CSV has a country column where Namibia (NA) shows up as missing after read_csv. Why, and how do you load it correctly?
How do missing values behave in pandas sums and means compared with NumPy, and what is pd.NA?
Expected output
country n_orders refund
0 NaN 5.0 NaN
1 IN 3.0 12.5
2 US NaN 0.0
missing: {'country': 1, 'n_orders': 1, 'refund': 1}
country n_orders refund
0 NA 5.0 NaN
1 IN 3.0 12.5
2 US NaN 0.0
missing: {'country': 0, 'n_orders': 1, 'refund': 1}
pandas sum/mean skip NaN: 10.0 5.0
NumPy does not: nan nan
sum(skipna=False): nan
[1, <NA>, 3]
[False, <NA>, True]
[False, True, False]