HectalZero → Async & the DOM

PHASE 4Beginner ~21 min· topic 2 of 6

Topic 4.2

Unicode, Emoji and String Length

In one line

Every character has a Unicode number (a code point), but JavaScript stores strings as 16-bit pieces called UTF-16 code units, so length counts pieces, not characters: one emoji can have a length of 2, 4 or more. Use for...of, spread or Intl.Segmenter to work with real characters, and normalize() before comparing text.

Think of it like this

Lego. Most letters are a small brick: one brick each. Some characters, like emoji, are too big for one brick, so they are built from two bricks clipped together. A family emoji is several figures joined with invisible clips. If you count bricks (length), a family looks like 8 things. If you count figures (code points), you get 5. A person looking at the screen sees 1 picture. Text tools must decide which of the three they're counting.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Unicode: The worldwide list that gives every character and emoji its own number.
Code point: The Unicode number of one character, written like U+0041 for A.
Encoding: A rule for storing code points as bytes or pieces, such as UTF-8 or UTF-16.
Code unit: One 16-bit piece of a JavaScript string. length counts these.
Surrogate pair: Two code units that together store one big code point, such as most emoji.
Lone surrogate: Half of a surrogate pair on its own. It isn't a valid character and shows as �.
Grapheme cluster: What a reader sees as one character, even if it's made of several code points.
Zero-width joiner (ZWJ): An invisible character (U+200D) that glues emoji together into one picture.
Normalisation: Turning text into one standard form, so things that look the same are stored the same.

Step by step

01Count a plain word

For everyday letters there's no surprise: one letter, one code unit. "Hello".length is 5, and "Hello"[1] is "e".

unicode.jswhole filejavascript

console.log("Hello".length);      // 5
console.log("Hello".codePointAt(0)); // 72, the code point of H

02Count an emoji

The grinning face is code point U+1F600 (128512 in decimal). That's bigger than 65535, so UTF-16 stores it as two code units: 0xD83D and 0xDE00. length counts both.

unicode.jswhole filejavascript

console.log("😀".length);          // 2
console.log("😀".charCodeAt(0));   // 55357 (0xD83D, half of the pair)
console.log("😀".codePointAt(0));  // 128512 (the real code point)
console.log([..."😀"].length);     // 1

Count an emojidiagram

Rendering diagram…

03Loop by code points

A classic for loop with i++ walks code units and splits the emoji in half. for...of walks code points, so each emoji arrives whole (unless it's a cluster like a family, which arrives as its parts).

unicode.jswhole filejavascript

for (const ch of "a😀b") console.log(ch); // a, 😀, b
// for (let i = 0; i < s.length; i++) would give: a, �, �, b

04Count what people see

Intl.Segmenter splits text into graphemes using Unicode's rules. segment() returns an iterable, so spreading it gives an array of segments. Its length is the number of visible characters.

unicode.jswhole filejavascript

const seg = new Intl.Segmenter("en", { granularity: "grapheme" });
const count = (s) => [...seg.segment(s)].length;
console.log(count("👨‍👩‍👧 hi")); // 4

05Compare text safely

Text from a Mac file name, a copy-paste or a different keyboard may arrive in NFD form. Normalise both sides before you compare them.

unicode.jswhole filejavascript

const a = "caf\u00e9";   // é as one code point
const b = "cafe\u0301";  // e + combining accent
console.log(a === b);                                 // false
console.log(a.normalize() === b.normalize());         // true

Try it yourself

1
Add your own samples
In the first example, add "\u{1F44B}\u{1F3FF}" (a waving hand with a skin tone) and the word "naïve". Predict the three numbers for each before you run it.
2
Cut without breaking
In the second example, add console.log(s.slice(0, 2)); and then console.log([...s].slice(0, 2).join(""));. Predict which one shows a broken character.
3
Measure bytes
Still in the second example, measure the UTF-8 bytes of "naïve" and of "नमस्ते" with TextEncoder. Which letters need more than one byte?

Code & diagrams

Three ways to count New tab

The last sample is the flag of India: two "regional indicator" letters, I and N, that the screen draws as one flag.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

cat 3 3 3
😀 2 1 1
👍🏽 4 2 1
👨‍👩‍👧 8 5 1
🇮🇳 4 2 1

Code units vs code points ES2024+ New tab

`isWellFormed` (ES2024) returns `false` when a string contains a lone surrogate, half an emoji. The UTF-8 count is 1 + 4 + 1 bytes.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

4
55357 56832
128512 1f600
true
4 3
false true
false b😀a
6 bytes in UTF-8

normalize, case and language ES2015+ New tab

Both words print as "café" but are stored differently. `\uFB01` is the single ligature character "ﬁ", which NFKC turns into the two letters "fi".

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

café café false
5 4
true
5
file
STRASSE
I İ

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Limit a name by length

Cut a display name to 5 "characters" with name.slice(0, 5) where name = "Riya👩‍💻".

terminal

$ the page shows

── what you'll see ──

Riya� — a broken replacement character after the name.

Break #2

Compare a pasted word

Check if (input === "café") where the user pasted café from a document saved in NFD form.

terminal

$ Console

── what you'll see ──

false (the check fails even though both look like café)

Myth vs fact

Myth

length is the number of characters.

Fact

It's the number of UTF-16 code units. It matches what you see only for text without emoji, combining marks and some rare scripts.

Myth

JavaScript strings are UTF-8.

Fact

Source files and network text are usually UTF-8, but inside the engine a string is a sequence of UTF-16 code units. Engines may store pure-Latin-1 strings with one byte per character internally, but the language still behaves as UTF-16.

Myth

for...of gives you visible characters.

Fact

It gives code points. A skin-tone emoji or a family still comes out in parts. Only Intl.Segmenter gives grapheme clusters.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
UTF-16 came from the 1990s idea that 65536 characters would be enough (UCS-2). When Unicode grew past that, surrogate pairs were added. JavaScript, Java and Windows all kept 16-bit strings for compatibility, which is why they share these problems.
▸
V8 stores strings that only contain characters up to U+00FF as one-byte strings internally and switches to two bytes per unit when a wider character appears. It's invisible to your code but affects memory use.
▸
Regular expressions match code units unless you add the u (or the newer v) flag: /^.$/.test("😀") is false, /^.$/u.test("😀") is true. With u you can also use property escapes like \p{L} (any letter) and \p{Emoji}. Topic 4.3 covers flags.
▸
Grapheme rules change as Unicode adds emoji, and Intl.Segmenter uses the browser's built-in ICU data, so very new emoji may segment differently in an old browser. Intl.Segmenter has worked in all major browsers since 2024 (Firefox was last).

Remember this

1
Unicode is the world's list of characters: every letter, digit, symbol and emoji in every language gets a number called a code point, written like U+0041 (that's A) or U+1F600 (😀). There is room for over a million code points, from U+0000 to U+10FFFF. Unicode only says which number means which character; an encoding says how to store those numbers as bytes. The web sends text as UTF-8 (1 to 4 bytes per code point), but inside JavaScript strings use UTF-16.
2
In UTF-16, text is stored in 16-bit pieces called code units (each holds a number from 0 to 65535). Code points up to U+FFFF (most letters in most languages, called the Basic Multilingual Plane) fit in one code unit. Bigger code points, including almost all emoji, need two code units called a surrogate pair: a "high" one (0xD800–0xDBFF) followed by a "low" one (0xDC00–0xDFFF). JavaScript's length, [i], charAt, charCodeAt, slice and split("") all work in code units. So "😀".length is 2 and "😀"[0] is half an emoji (a lone surrogate, which shows as �).
3
To work with code points, use the tools added in ES2015: for...of loops over a string one code point at a time, and so do spread [...str] and Array.from(str). codePointAt(i) reads the full number at a position, String.fromCodePoint(n) builds a character from it, and the escape \u{1F600} writes any code point in source code (the older \u0041 form takes exactly four hex digits, so it can only write one code unit). [...str].length counts code points.
4
Even code points aren't what a person calls "a character". A grapheme cluster is what a reader sees as one character: 👍🏽 is a thumbs-up plus a skin-tone modifier (2 code points, 4 code units), and 👨‍👩‍👧 is three people joined by zero-width joiners (ZWJ, U+200D): 5 code points, 8 code units, 1 visible picture. Flags are two regional-indicator letters. To count or cut by what the user sees, use **Intl.Segmenter** with granularity: "grapheme"; it follows Unicode's segmentation rules. It's the right tool for a "characters left" counter or for cutting a long name without breaking an emoji.
5
The same visible text can be stored in more than one way. é can be one code point (U+00E9, "precomposed") or two: e followed by a combining acute accent (U+0301). They look identical but === says they're different, and their lengths differ. str.normalize() (ES2015) converts text to a standard form: NFC (the default) composes characters where possible, NFD splits them apart. Normalise to NFC before comparing, searching or storing user text. The NFKC form also folds look-alike "compatibility" characters, such as the ligature ﬁ into fi.
6
Other places the gap between code units and real characters shows up: reversing a string with split("").reverse().join("") breaks emoji (spread first instead); toUpperCase can change the length ("ß".toUpperCase() is "SS") and depends on language ("i".toLocaleUpperCase("tr") is "İ" in Turkish); and a database column or an API limit usually counts UTF-8 bytes, which new TextEncoder().encode(str).length measures (😀 is 4 bytes). ES2024 added isWellFormed() and toWellFormed() to find and repair lone surrogates before you send text to an API that would reject it.

Explain it without notes

01

Why is "😀".length equal to 2?

02

What is the difference between a code unit, a code point and a grapheme cluster?

03

How would you count the characters a user sees in a text box?

04

Why can two strings that look identical fail ===, and how do you fix it?

Practice

01

Write reverseText(s) that reverses a string without breaking emoji like 😀 (code points are enough here).

02

Write charsLeft(text, max) that returns how many visible characters are left, counting graphemes, so charsLeft("hi👍🏽", 10) returns 7.

Trade-offs

↔
Intl.Segmenter is correct for what people see but slower than length; use length for internal limits and protocol work, and graphemes for anything a person counts.
↔
Normalising on input (store NFC everywhere) costs a little at write time but makes every later comparison and search simple; normalising at every comparison is easy to forget.

Done when you can

I can explain code units, code points and grapheme clusters with an emoji example.
I know length, [i], slice and split("") work in UTF-16 code units.
I can loop over code points with for...of and spread a string safely.
I can count visible characters with Intl.Segmenter.
I normalise text with normalize() before comparing it.