PHASE 4Beginner ~26 min· topic 2 of 7

Topic 4.2

Loading CSV and JSON

In one line

Data arrives as text over a network that can fail. fetch and d3.csv bring it in, but you must check the answer, convert text to numbers and dates on purpose, and show loading, empty and error states instead of a blank chart.

Think of it like this

You order a parcel. You give an address, and the courier comes back with one of four things: the parcel, "no such address", nobody at home, or a box with the wrong thing inside. Only after you open the box and check it do you use what is in it. And everything in the box is wrapped in plain paper: text. Loading data is the same: ask, check the answer, open it, check what is inside, and then use it.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

fetch: The browser's function for requesting a URL. It returns a promise of a Response.
response.ok: True when the status code is 200 to 299. It is false for 404, 500 and the like, even though fetch itself did not fail.
status code: A number the server sends back: 200 means fine, 404 means not found, 500 means the server broke.
parse: Turn text into structured data, for example CSV text into a list of row objects.
header row: The first line of a CSV, holding the column names.
row function: A function you give d3.csv that turns each text row into the object you want (numbers, dates).

Step by step

01One request, many outcomes

A request can end in five different places, and each needs its own handling. The diagram follows one fetch of daily.csv from the request to a chart.

One request, many outcomesdiagram

Rendering diagram…

02Meet the strings

The first example prints the file's first two lines, then typeof rows[0].rides is string, and "1193" + 1 is 11931. The biggest day as text is 910, as numbers 4380.

03Check what autoType does to awkward cells

The second example feeds d3.autoType fourteen cells like the ones in Lumen's files. Read the table: S01 stays text, 07790 and 2025 become numbers, 2,016, n/a and 13/03/2025 stay text, an empty cell becomes null, and 2025-03-32 becomes an Invalid Date.

The last two lines show the time-zone trap: 2025-01-01 becomes midnight UTC, which on Ardenvale's winter clock (one hour behind UTC) is 23:00 on 31 December. Topic 4.3 goes deeper into dates.

04Run eight inputs through one loader

The third example is a loader that handles every outcome in the diagram, and a box that always shows exactly one state. It runs eight inputs through it: a good file, a wrong path, a header-only file, a blocked address, a CSV read as JSON, two web pages (one read as JSON, one as CSV), and a JSON list.

Read the console lines: the wrong path is an Error we made from response.ok; the blocked address is a TypeError from fetch; both bad-JSON cases are a SyntaxError with the first characters of the text in the message; the web page read as CSV is caught only by the column check.

Try it yourself

1
Point the first example at a missing file
In the first example change daily.csv in the fetch line and in d3.csv to nope.csv. The first print shows the body of the 404 answer, not found, as if it were data, because fetch did not fail. Then d3.csv rejects, and the Console ends with Uncaught Error: 404 Not Found.
2
End the loader on a failure
In the third example move the line await show("a wrong path", …) to the very end. The box on the page now stays red: Could not load a wrong path. the server answered 404 for /data/lumen/nope.csv. The page shows the last state, the console shows them all.
3
Read a JSON object as if it were a list
In the third example change the last line to await show("status.json", "/data/lumen/status.json", "json");. The console prints status.json data (undefined rows). status.json is an object ({ updated, stations }), so it has no .length, and undefined === 0 is false. The loader reports success with nothing to draw. Fix it with Array.isArray(data), or read data.stations.

Code & diagrams

Fetch a CSV and look at what you got New tab

This is a module (it has an import), so top-level await works. In a classic script, wrap the code in an async function.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

First 2 lines of the file:
date,rides,ebike_rides,avg_minutes,temp_c,rain_mm,weekday
2025-01-01,1193,346,16.5,12.9,7.8,Wed
Rows: 365 | same with d3.csv: 365 | columns: date,rides,ebike_rides,avg_minutes,temp_c,rain_mm,weekday
First row: { date: '2025-01-01', rides: '1193', ebike_rides: '346', avg_minutes: '16.5', temp_c: '12.9', rain_mm: '7.8', weekday: 'Wed' }
typeof rides: string | rides + 1 = 11931 (joined, not added)
Biggest day as text: 910 | as numbers: 4380

What d3.autoType does to cells like Lumen's New tab

Failure cases that print no error: the leading zero vanishes, the comma number stays text, and the impossible date is an Invalid Date rather than an exception.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

"S01"         -> string  S01
"07790"       -> number  7790
"7790"        -> number  7790
"2,016"       -> string  2,016
"n/a"         -> string  n/a
""            -> null    null
"-1"          -> number  -1
"0x1A"        -> number  26
"1e3"         -> number  1000
"2025"        -> number  2025
"true"        -> boolean true
"2025-01-01"  -> Date    2025-01-01T00:00:00.000Z
"2025-03-32"  -> Date    Invalid Date
"13/03/2025"  -> string  13/03/2025
2025-01-01 as a Date: 2025-01-01T00:00:00.000Z
The same instant on Ardenvale's wall clock: 31 Dec 2024, 23:00

One loader, four page states, eight inputs New tab

Only the last state stays on the page (a green box here), but the console lists all eight. The header-only file is empty, not an error. "another website" is blocked in this preview, like any address outside the site.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

daily.csv               data (365 rows)
a wrong path            error: Error: the server answered 404 for /data/lumen/nope.csv
a header-only file      empty
another website         error: TypeError: Failed to fetch
a CSV read as JSON      error: SyntaxError: Unexpected token 'd', "date,rides"... is not valid JSON
a web page read as JSON error: SyntaxError: Unexpected token '<', "<!doctype "... is not valid JSON
a web page read as CSV  error: Error: not the file we wanted: no column date, rides
bikes.json              data (320 rows)

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Trust the numbers without converting

In the first example, add this line before the last one: console.log("Extent:", d3.extent(rows, (d) => d.rides));

terminal

$ Console

── what you'll see ──

Extent: [ '1001', '910' ]

Break #2

Let autoType turn a code into a number

In the second example, change "07790" in cells to "007790" and read the first two number rows.

terminal

$ Console

── what you'll see ──

"007790" -> number 7790

Break #3

Check only response.ok on a server that answers 200 for everything

On your own computer, change DATA_URL in a project that loads daily.csv to "/data/lumn/" (one letter missing) and load the file with only an ok check.

terminal

$ Console (Vite dev server)

── what you'll see ──

TypeError: Cannot read properties of undefined (reading 'trim')

Myth vs fact

Myth

If fetch does not throw, the file loaded.

Fact

fetch resolves for a 404 and a 500 too. Only response.ok tells you the server found the file, and only a column check tells you it is the right file.

Myth

d3.csv gives me numbers.

Fact

It gives strings, unless you add d3.autoType or a row function. A chart built on strings shows a blank, NaN or a wrong maximum.

Myth

d3.autoType is safe for any file.

Fact

It turns 07790 into 7790, treats 2025 as a number, leaves 2,016 as text, and makes date-only text a UTC midnight. Use it to explore, and write a row function for anything you ship.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
d3.csv(url, init) passes init to fetch, so you can send headers or credentials. D3 7 uses fetch underneath. d3.csv rejects with a plain Error whose message is the status line (404 Not Found), not a Response, so you cannot read the body of an error answer through it. Use fetch yourself when the server explains its errors in the body.
▸
d3.autoType reads a date-time without a zone (2025-06-01T00:00, as in hourly-june.csv) as local time of the visitor's computer, but a date-only text (2025-06-01) as UTC. D3 adds a workaround so both behave alike in browsers that parse them differently. The same column can therefore mean different instants for two visitors. Topic 4.3 says which clock each Lumen file is on.

Remember this

1
The problem this solves: a chart needs numbers, dates and a list of objects, but a file arrives as one long text over a connection that can fail. If you skip the checks, a missing file gives a blank chart, a wrong type gives NaN or a wrong maximum, and the person looking at the page sees no explanation at all.
2
**fetch succeeds even when the file is missing.** fetch(url) returns a promise that rejects only when no answer could be obtained: the network is down, the address is blocked, or the browser forbids the request. Then you get TypeError: Failed to fetch. If the server answers at all, even with 404 Not Found, the promise resolves with a Response. You must look at response.ok (true for status 200 to 299) yourself. Reading the body can fail too: response.json() throws a SyntaxError when the text is not valid JSON.
3
CSV and JSON. CSV is plain text: the first line holds the column names (the header row) and every later line is one row, with commas between cells. A cell that contains a comma is wrapped in quotes, like "2,016". JSON keeps real types: numbers, strings, true, null, lists and nesting. JSON must be exactly valid (a trailing comma breaks it). A JSON file can hold a list or a single object, so check Array.isArray before you use .length.
4
**d3.csv and d3.json do the common steps.** await d3.csv(url) fetches, checks the status, reads the text and returns a list of objects (with a columns list of the header names). For a status that is not 2xx it rejects with Error: 404 Not Found. Network failures still give TypeError: Failed to fetch. You may see older code that passes a callback (d3.csv(url, function(error, data) {…})): that is D3 version 4 and earlier, and no longer works in D3 7.
5
Everything in a CSV is a string. "1193" + 1 is "11931", and d3.max over the strings "4380" and "910" returns "910" because text sorts letter by letter. Convert on purpose: +d.rides or Number(d.rides) per column, or give d3.csv a row function that returns a clean object. d3.autoType guesses a type for every cell and is handy for exploring, but its guesses have traps (see the second example): "07790" becomes the number 7790 and loses its zero, "2025" becomes a number, "2,016" stays text, a date-only text becomes a Date at midnight UTC, and "2025-03-32" becomes an Invalid Date without any error.
6
Some servers answer 200 for a wrong path. Vite's dev server (and many hosts for single-page apps) returns the index.html page for paths it does not know. response.ok is true, the body is HTML, and a CSV parser happily turns it into nonsense rows. After parsing, check that the columns you need are really there.
7
Show every state on the page. A chart has at least four states: loading (the request is running), error (it failed: say what and what to try), empty (it worked but there are no rows: that is not an error and needs its own message), and data. Always show exactly one of them. The third example sets all four with one loader and runs eight different inputs through it.

Explain it without notes

Why does fetch of a missing file not throw, and what must you check?

What are the four states a data-driven chart should handle, and what does each one show?

Give three ways d3.autoType can silently go wrong on Lumen-like data.

Practice

Write loadCsv(url, columns) that returns the rows of a CSV, throws Error("…") with a clear message for a non-OK status, and throws when any of columns is missing from the header. Test it with /data/lumen/stations.csv and the columns ["id", "name"].

Load stations.csv with a row function so that id, name and district stay text, lon, lat and docks are numbers and opened is a Date. Print the number of stations and the docks of the first one.

Trade-offs

↔
d3.autoType saves typing and can ship a quiet bug; a row function costs one line per column and documents what each column means. Use autoType while exploring, a row function in the dashboard.

Done when you can

I can say what makes fetch reject and what only response.ok can tell me.
I can load a CSV and a JSON file and say what type every value has.
I can name three d3.autoType traps and what I use instead.
I can show loading, error, empty and data states on a page, and keep them separate.
I check that parsed data has the columns I expect.

Command Palette

Words you'll meet

Step by step

01One request, many outcomes

02Meet the strings

03Check what autoType does to awkward cells

04Run eight inputs through one loader

Try it yourself

Point the first example at a missing file

End the loader on a failure

Read a JSON object as if it were a list

Code & diagrams

Break it on purpose

Trust the numbers without converting

Let autoType turn a code into a number

Check only response.ok on a server that answers 200 for everything

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can