PHASE 3Beginner ~27 min· topic 5 of 7

Topic 3.5

Immutability, Interning and Building Strings Fast

In one line

A string can never be changed after it is created: every "change" builds a new string. Knowing this explains why s[0] = "J" fails, why joining pieces beats += in a loop, why you should compare strings with == and never is, and when io.StringIO is the right buffer.

Think of it like this

A printed book. You cannot change a word in a printed page; if you want a different sentence, you print a new page. A Python string is printed text. "hello" is fixed forever, and what looks like an edit (s.upper(), s + "!", s.replace(...)) is Python printing a new page and leaving the old one alone. The word for "cannot be changed after creation" is immutable.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Immutable: Cannot be changed after it is created. Strings, numbers and tuples are immutable.
Mutable: Can be changed in place. Lists and dictionaries are mutable (Phase 4).
Concatenate: Join two strings end to end with +.
Interning: Keeping one shared copy of a string and reusing it, instead of storing equal copies separately.
Identity: Whether two names point to the very same object in memory. Tested with is.
Equality: Whether two values hold the same content. Tested with ==.
Buffer: A place where pieces of data are collected before the final result is used.
StringIO: A text file that lives in memory, made with io.StringIO().

Step by step

01Every change makes a new string

Look at what happens to s and to u, a second name for the same string. When s is rebuilt as s + " world", a new string appears and s now points at it. u still points at the old "hello", which was never touched. The methods work the same way: they return new strings.

main.pywhole filepython

s = "hello"
t = s.upper()
print(s, t)                 # hello HELLO

u = s
s = s + " world"
print("s:", s, "| u:", u)   # s: hello world | u: hello

Every change makes a new stringdiagram

Rendering diagram…

02You cannot assign into a string

Square brackets can read a character but not write one. If you need to change one letter, build a new string from the parts around it, or replace by value with replace.

main.pywhole filepython

s = "hello"
fixed = "J" + s[1:]        # new string: J + ello
print(fixed)               # Jello
print(s.replace("h", "J")) # Jello  (again a new string)

03Building a string in a loop: the slow pattern

sentence += w + " " looks harmless. Each step creates a new string and, in general, copies everything built so far. For a few dozen pieces nobody notices; for a hundred thousand lines of log text, it becomes the slowest part of the program.

main.pywhole filepython

words = "to be or not to be".split()

sentence = ""
for w in words:
    sentence += w + " "     # a new string every time round

04The fast pattern: collect, then join once

A list (Phase 4) is a row of items you can add to. pieces.append(x) adds x at the end. A list of pieces grows cheaply, because adding to a list does not copy the pieces. "".join(pieces) then measures everything, makes one result string and copies each piece in once. The cost grows in step with the total text, not with its square.

If your pieces already exist as a list, you do not even need the loop: " ".join(words) is the whole job.

main.pywhole filepython

pieces = []
for n in range(5):
    pieces.append(str(n * n))   # numbers must become strings first

print(",".join(pieces))          # 0,1,4,9,16

The fast pattern: collect, then join oncediagram

Rendering diagram…

05Equality is not identity

Two equal strings are not always the same object. Building a string at run time (here, with join) creates a fresh object each time. They are == but not is. After sys.intern, both names point to the single shared copy.

The lesson is not to memorise when Python interns. The lesson is that is on strings is the wrong question. Use == for text. The only common use of is is is None.

main.pywhole filepython

import sys

a = "".join(["py", "thon"])
b = "".join(["py", "thon"])
print(a == b)        # True   same text
print(a is b)        # False  two separate objects

print(sys.intern(a) is sys.intern(b))   # True

06StringIO: a file that lives in memory

io.StringIO() gives you an object with write, so anything that can print or write to a file can write into it. print(..., file=buffer) sends the output of print there. Call getvalue() to get the whole text back as one string.

A common use is testing: capture what a function prints, or hand a fake file to a function that expects a real one.

main.pywhole filepython

import io

buffer = io.StringIO()
buffer.write("line 1\n")
print("line 2", file=buffer)

print(repr(buffer.getvalue()))   # 'line 1\nline 2\n'

07When you really need a changeable text

If you need to change individual characters (for example to flip letters in an algorithm), turn the string into a list of characters with list(s), change the list, then join it back with "".join(chars). You will use this pattern in the DSA course (/dsa) for string problems. Lists are mutable; Phase 4 covers them.

Try it yourself

1
Two names, one string
Run the first example, then change u = s to u = s[:] and s = s + " world" to s += " world". The output is the same. Explain why: += on a string does not change the old string. It builds a new string and points the name s at it (programmers say it rebinds s).
2
Count the pieces
Change range(5) to range(20) in the join example and print the result. It works the same for 20 or 20 million pieces; only the speed changes.
3
Never use is for text
In the third example, change the text to "hello", built from "he" + "llo", and print a is b again. Whether it prints True or False, do not rely on it. The right test for text is always ==.

Code & diagrams

Every change makes a new string New tab

Expected output

hello HELLO
s: hello world | u: hello
hello there | s is still: hello world

Join instead of += in a loop New tab

Both ways give the same text. Only the cost differs, and the difference grows with the size of the input.

Expected output

'to be or not to be '
'to be or not to be'
same text: True
0,1,4,9,16

Equal is not the same object New tab

Expected output

equal: True
same object: False
after interning, same object: True

StringIO as an in-memory file New tab

Expected output

line 1
line 2
line 3
line 4
characters: 28

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Assign to a position in a string

Run s = "hello" and then s[0] = "J".

main.pywhole filepython

s = "hello"
s[0] = "J"

terminal

$ python main.py

── what you'll see ──

Traceback (most recent call last):

File "main.py", line 2, in <module>

s[0] = "J"

~^^^

TypeError: 'str' object does not support item assignment

Myth vs fact

Myth

s += "x" appends to the string in place.

Fact

It builds a new string and rebinds s. (CPython sometimes reuses memory behind the scenes, but the old value is never visible changed to anyone else.)

Myth

is is a faster == for strings.

Fact

They answer different questions. is asks if two names point to the very same value in memory; the result for equal strings depends on how they were created. Use == for text.

Myth

Immutable means the variable cannot be reassigned.

Fact

The variable can point somewhere else at any time. It is the string object that cannot change.

Myth

Joining is only a style choice.

Fact

For large inputs it changes the cost from roughly quadratic to linear in the worst case, and it works the same on every Python implementation.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
CPython's in-place += optimisation applies when the left string has a reference count of one, so x = x + y or x += y on a local variable can use realloc. It breaks as soon as a second name or a container holds the string, which is why benchmark results for "+= is fast" do not carry over to real programs, or to PyPy. Write join.
▸
Strings cache their hash after the first computation, and equal strings must have equal hashes. That is why a string can be a dictionary key: once stored, the key cannot change underneath the table (Topic 4.5). Immutability is what makes hash(s) stable for the whole life of the object.
▸
sys.intern matters in code that holds millions of repeated strings (column names, tags, tokens): interned copies save memory, and comparing two interned strings can short-circuit on identity before looking at the characters. Compile-time constants that look like identifiers are interned automatically, so a is b between two such literals may be True. That is an implementation detail and can change between versions.

Remember this

1
Trying to change a character in place fails: s[0] = "J" raises TypeError: 'str' object does not support item assignment. To "change" a string you build a new one from pieces ("J" + s[1:]) and point the variable at it. Other names that still point at the old string are not affected, which makes strings very safe to share between parts of a program.
2
Immutability is not a limitation Python regrets; it buys three things. Strings can be shared freely without copying, they can be used as dictionary keys (Phase 4: a key must never change after it is stored), and Python can cache them. The price is that building a long string one piece at a time creates a lot of short-lived strings.
3
Consider result += piece in a loop. Each += makes a brand-new string of the combined length and copies all the old text into it, so the work grows roughly with the square of the number of pieces.
4
CPython has a special trick that often resizes the string in place when nothing else points to it, which is why such loops can look fast. It is an implementation detail, not a promise, and other Python implementations do not have it. The portable, fast pattern is: collect the pieces in a list, then call "".join(pieces) once.
5
Interning means keeping one shared copy of a string instead of many equal copies. CPython interns many strings automatically: the empty string, one-character strings, and strings in your code that look like names. Because of this, a is b sometimes happens to be True for equal strings, and sometimes False. Both are normal. is asks "is this the very same value in memory?", not "is the text equal?". To compare text, **always use ==**. (Python even warns you if you write x is "hello".)
6
You can intern a string yourself with sys.intern(s), which returns the one shared copy. It is a specialist tool for programs that compare an enormous number of repeated strings, because comparing two interned strings can be a quick identity check. Most programs never need it.
7
io.StringIO is an in-memory text file. You write to it or print(..., file=buffer) into it, and call getvalue() at the end. It is the right tool when code expects a file (a CSV writer or a logging handler, both from Phase 7, or a function that prints) but you want the text in a variable, and it is also a good way to build big output piece by piece.

Explain it without notes

What does it mean that strings are immutable, and what happens when you write s = s + "!"?

Why is "".join(pieces) preferred over += in a loop?

Why is a == b the right way to compare strings, and what can a is b give?

When would you use io.StringIO?

Practice

Build the string "01234" once with += in a loop and once by collecting pieces in a list and joining them. Print both and whether they are equal.

Use io.StringIO and print(..., file=buffer) to collect three lines row 0, row 1, row 2 and then print the collected text.

Make two equal strings using "he" + "llo" and sys.intern("hel" + "lo") and show that comparing with == is True.

Trade-offs

↔
Safety against speed. Immutable strings are safe to share and to use as keys, but building text piece by piece needs a buffer (list or StringIO) to stay fast.
↔
Clarity against micro-optimisation. For ten pieces, + or an f-string is clearest. Switch to join when the number of pieces grows with the data, such as one per line of a file.

Done when you can

Done when you can explain immutability using the printed-page idea and show what s += "x" really does.
Done when you can build a large string with a list and join.
Done when you can explain why is is wrong for comparing strings.
Done when you can say what sys.intern does.
Done when you can capture text with io.StringIO.

Command Palette

Words you'll meet

Step by step

01Every change makes a new string

02You cannot assign into a string

03Building a string in a loop: the slow pattern

04The fast pattern: collect, then join once

05Equality is not identity

06StringIO: a file that lives in memory

07When you really need a changeable text

Try it yourself

Two names, one string

Count the pieces

Never use is for text

Code & diagrams

Break it on purpose

Assign to a position in a string

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can