Command Palette

Search for a command to run...

Hectal
PHASE 4Beginner ~27 min· topic 9 of 11

Topic 4.9

References, Shallow Copy and Deep Copy

In one line

Assigning a list or dictionary to another name does not copy it; both names point at the same object. A shallow copy makes a new outer container that still shares the inner objects, and a deep copy duplicates everything all the way down.

Think of it like this

A shared online document. If you send a friend the link, you both edit the same document (an alias). If you download a copy and edit it, your friend's version is unchanged (a copy). But if the document contains a link to a spreadsheet, and you download only the document, both copies still open the same spreadsheet (a shallow copy). Only a full copy of the document and everything it links to is independent (a deep copy).

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Reference
An arrow from a name, or from a slot inside a container, to an object.
Alias
A second name for the same object. Made by b = a.
Identity (`is`)
Whether two names point at the very same object.
Equality (`==`)
Whether two objects have equal contents.
Shallow copy
A new outer container whose slots point at the same inner objects.
Deep copy
A copy of the container and, recursively, of everything inside it.
Snapshot
A copy that records the data as it is now, unaffected by later changes.
Cycle
A structure that contains itself, directly or through other objects.

Step by step

01Assignment makes an alias

b = a creates no new list. After the assignment a is b is True. This is useful (passing big data around costs nothing) and dangerous (an unexpected change).

main.pywhole filepython
a = [1, 2, 3]
b = a
b.append(4)
print(a)          # [1, 2, 3, 4]
print(a is b)     # True
Assignment makes an aliasdiagram
Rendering diagram…

02A shallow copy: new outside, shared inside

The outer lists are different objects, so appending to one outer list does not touch the other. But both outer lists hold arrows to the same inner lists.

main.pywhole filepython
original = [[1, 2], [3, 4]]
shallow = original.copy()
shallow.append([5, 6])        # only the copy grows
shallow[0].append(99)         # the SHARED inner list grows
print(original)               # [[1, 2, 99], [3, 4]]
A shallow copy: new outside, shared insidediagram
Rendering diagram…

03A deep copy shares nothing

copy.deepcopy walks through the structure and copies every container it finds. After that, no change to the copy can reach the original.

main.pywhole filepython
import copy

original = [[1, 2], [3, 4]]
deep = copy.deepcopy(original)
deep[0].append(99)
print(original)    # [[1, 2], [3, 4]]  unchanged
print(deep)        # [[1, 2, 99], [3, 4]]

04The multiplied-rows trap

[0] * 3 makes a list of three zeros. Zeros are immutable, so repeating them is fine. But [[0] * 3] * 3 repeats a list, and the three arrows point at one list. Use a comprehension so that each row is built separately.

main.pywhole filepython
bad = [[0] * 3] * 3
bad[0][0] = 1
print(bad)       # [[1, 0, 0], [1, 0, 0], [1, 0, 0]]

good = [[0] * 3 for _ in range(3)]
good[0][0] = 1
print(good)      # [[1, 0, 0], [0, 0, 0], [0, 0, 0]]

05Changing a list while looping over it

The loop keeps an internal position. Removing an item moves the later ones down by one, so the item that slid into the current position is never looked at. The cure is to loop over something that is not changing.

main.pywhole filepython
nums = [2, 4, 6, 8]
for n in nums:
    if n % 2 == 0:
        nums.remove(n)
print(nums)      # [4, 8]   not empty!

nums = [2, 4, 6, 8, 9]
nums = [n for n in nums if n % 2 != 0]   # build a new list instead

06Which copy do I need?

If the container holds only numbers, strings or tuples: a shallow copy is enough. If it holds lists, dictionaries or other changeable objects and you will change those inner objects: use deepcopy. If you only read: no copy is needed at all.

Try it yourself

  1. 1

    Draw it first

    In the first example, draw the arrows for original, shallow and deep on paper, then predict the four is lines. Run it to check.

  2. 2

    Make the bug and cure it

    In the second example, add the line print(bad[0] is bad[1]) right after bad[0][0] = 1 and run it. True is the proof that the rows are one list. Then change good to use the comprehension in bad (replace [[0] * 3] * 3 with [[0] * 3 for _ in range(3)]), run again, and compare the two grids.

Code & diagrams

Alias, shallow copy and deep copy side by side New tab

Look at which of the three results changed when the inner list was extended.

Sign in to run this example in your browser.

Expected output

alias is original: True
shallow is original: False
shallow[0] is original[0]: True
deep[0] is original[0]: False
after append -> alias: [[1, 2], [3, 4], [5, 6]]
after append -> shallow: [[1, 2], [3, 4]]
after inner change -> shallow: [[1, 2, 99], [3, 4]]
after inner change -> deep: [[1, 2], [3, 4]]
dict shallow copy shares the list: [10, 12, 14]
The repeated-row trap and changing a list in a loop New tab
Sign in to run this example in your browser.

Expected output

bad grid: [[1, 0, 0], [1, 0, 0], [1, 0, 0]]
good grid: [[1, 0, 0], [0, 0, 0], [0, 0, 0]]
loop and remove: [4, 8]
loop over a copy: [9]
comprehension: [9]
+= changes the list, a + b makes a new one New tab

Two statements that look alike behave differently when another name points at the same list. A tuple has no in-place version, so it never surprises you this way.

Sign in to run this example in your browser.

Expected output

after a += [3]: [1, 2, 3] [1, 2, 3] | same list: True
after a = a + [4]: [1, 2, 3, 4] [1, 2, 3] | same list: False
tuple: (1, 2, 3) (1, 2)

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Edit a shallow copy and damage the original

Change an inner list of a shallow copy.

main.pywhole filepython
team = {"name": "A", "members": ["Asha", "Ravi"]}
backup = team.copy()
backup["members"].append("Tom")
print("team:", team)
terminal
$ python main.py
── what you'll see ──
team: {'name': 'A', 'members': ['Asha', 'Ravi', 'Tom']}

Break #2

Remove from a list while looping

Delete every even number inside the loop.

main.pywhole filepython
nums = [2, 4, 6, 8]
for n in nums:
    if n % 2 == 0:
        nums.remove(n)
print(nums)
terminal
$ python main.py
── what you'll see ──
[4, 8]

Myth vs fact

Myth

copy() copies everything.

Fact

It copies only the outer container. Inner lists and dictionaries are shared (a shallow copy).

Myth

I should always use deepcopy to be safe.

Fact

Deep copies are slow and use more memory, and are unnecessary when the items are immutable or you never change them. Choose on purpose.

Myth

[[0] * 3] * 3 makes three separate rows.

Fact

It makes one row repeated three times. Use [[0] * 3 for _ in range(3)].

Myth

Strings and numbers are copied when I assign them.

Fact

They are shared too. It never matters, because they cannot be changed; any "change" builds a new object.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

  • ▸

    Python passes object references to functions, so a function that mutates a list argument changes the caller's list. Be explicit: either document that you modify, or copy at the start (items = list(items)). Phase 5 returns to this.

  • ▸

    copy.deepcopy keeps a memo dictionary of objects already copied, which is how it handles cycles and shared sub-objects (copying a structure where two slots point at one inner list gives two slots pointing at one new inner list). You can customise it per class with __deepcopy__.

  • ▸

    Deep copying is a common hidden performance cost. A cheaper pattern for config or messages is to treat data as read-only and make a new object for each change. Libraries for immutable data and frozen dataclasses (Topic 8.8) support this style.

  • ▸

    is compares identity, == compares contents (Topic 1.8, and Topic 13.1 goes deeper). Never use is to compare numbers or strings: CPython caches small ints (-5 to 256) and may reuse equal strings, so a is b can be True for equal values by accident, and False for the same values elsewhere. That is an implementation detail, not a promise. Use is only for None, True, False and for a deliberate "same object?" question. Another subtle point: deepcopy returns atomic immutable values (numbers, strings, None), and tuples whose items are all unchanged, as they are, because copying them would change nothing.

  • ▸

    In AI code, a prompt template shared by many requests is a classic victim: one request appends to the shared messages list and every later request sees the extra messages. Copy the template per request (list(template) is enough when the messages are never mutated, deepcopy when they are).

Remember this

  1. 1

    Python names are arrows to objects (Topic 1.1). b = a copies the arrow, not the object. For numbers and strings this never matters, because they cannot be changed. For lists, dictionaries and sets, which can change, two names for one object means a change through one name is visible through the other. a is b tells you whether two names point at the very same object, while a == b only compares contents.

  2. 2

    A shallow copy creates a new outer container and copies the arrows inside it. For a list: a.copy(), a[:] or list(a); for a dictionary: d.copy(), dict(d) or {**d} (which spreads the pairs of d into a new dictionary); for a set: s.copy() or set(s). Changing the outer container (appending, removing, replacing an item) now affects only the copy. But the inner objects are shared: if a holds a list, a.copy() holds an arrow to the same inner list.

  3. 3

    That sharing is the source of the classic bug. After b = a.copy() where a = [[1, 2], [3, 4]], running b[0].append(99) also changes a[0], because b[0] and a[0] are the same inner list. A shallow copy is perfectly fine when the items are numbers, strings or tuples of those, because none of them can be changed.

  4. 4

    A deep copy makes a new copy of the outer container and also of everything inside it, however deep: import copy then copy.deepcopy(a). (copy is a toolbox that comes with Python; Topic 6.1 teaches import.) The copy shares nothing changeable with the original. It is slower and uses more memory. It also copes with a list that contains itself, so it does not loop forever. Use it when you need a fully independent snapshot of nested data.

  5. 5

    The most famous trap: grid = [[0] * 3] * 3. The inner [0] * 3 is evaluated once, and the outer * 3 repeats the same arrow three times. So all three rows are one list, and grid[0][0] = 1 changes every row. The correct way is a comprehension, which evaluates the inner expression fresh for each row: [[0] * 3 for _ in range(3)].

  6. 6

    The second trap is changing a list while looping over it. Python walks the list by position. If you remove an item, later items slide left and the loop skips the next one. Loop over a copy (for n in nums[:]), or build a new list with a comprehension, or loop over the indexes backwards. For dictionaries and sets, changing their size during a loop raises RuntimeError (Topic 4.4).

Explain it without notes

01

What happens when you write b = a for a list, and how do you get a separate list?

02

What is the difference between a shallow copy and a deep copy?

03

Why does [[0] * 3] * 3 behave surprisingly, and how do you fix it?

04

Why is it a bug to remove items from a list inside for item in that_list?

05

After b = a, what is the difference between a += [3] and a = a + [3] for lists, and why does it matter?

Practice

01

Make a = [1, 2, 3]. Create b as a real copy (not an alias), append 4 to b, and print both lists.

02

Make data = {"tags": ["a", "b"]}. Make a deep copy, append "c" to the copy's tags, and print both to prove they are independent.

03

Build a 2-row, 3-column grid of zeros correctly, set the first cell of the first row to 5, and print it.

Trade-offs

  • ↔

    Sharing is cheap, copying is safe. Passing the same list around costs nothing but allows surprising changes; copying protects you but costs time and memory in proportion to the size.

  • ↔

    Shallow copy is fast and often enough. Deep copy is the only fully safe snapshot, but it walks the entire structure. Choose by what you will change.

Done when you can

  • Done when you can explain b = a for a list using arrows and is.

  • Done when you can name three ways to make a shallow copy of a list and of a dictionary.

  • Done when you can show a shallow copy sharing an inner list and fix it with deepcopy.

  • Done when you can explain and fix the [[0] * 3] * 3 trap.

  • Done when you can remove items from a list safely while processing it.