Special Methods: The Python Data Model

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Special (dunder) method

A method with double underscores that Python calls for you in response to syntax or a built-in function.

Data model

The full set of special methods that define how objects work with Python's syntax and built-ins.

`__repr__`

Returns developer-facing text for an object, ideally valid code that rebuilds it.

`__str__`

Returns user-facing text for an object. Used by print and str().

`NotImplemented`

A special value returned by an operator method meaning "I do not know how to handle this type", so Python tries the other side.

Reflected method

A method like __radd__ that Python tries on the right operand when the left one cannot handle the operation.

Callable

Anything you can call with brackets: functions, classes, and objects with __call__.

Hashable

Can be used as a dictionary key or set member. Needs __hash__ and a matching __eq__.

Truthiness

Whether an object counts as True or False in if. Decided by __bool__, then __len__.

1. 1

   Print three ways

   In the first example, comment out __str__ and run: print(a) now shows the repr text. Then comment out __repr__ too and you get the ugly default with a memory address (do this on your own machine, not for the output check).

2. 2

   Add subtraction

   Add __sub__ so a - b works and prints <2, 2>. Then ask yourself: do you need __rsub__? Only if the left side could be a type that does not know about vectors.

3. 3

   Make the playlist truthy

   Add def __bool__(self): return True to Playlist and print bool(Playlist("empty", [])). An explicit __bool__ beats __len__.

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __eq__(self, other):
        return self.x == other.x and self.y == other.y


seen = {Point(1, 2)}

class Vector:
    def __init__(self, x, y):
        self.x = x
        self.y = y


print(Vector(1, 2) + Vector(3, 4))

1. 1

   Special methods are also called dunder methods (double underscore, like __len__). You almost never call them directly. Python does it: len(p) calls p.__len__(), and a + b calls a.__add__(b). The full set of these hooks is called the Python data model, and it is why user-made classes can feel as natural as lists and numbers.

2. 2

   __repr__ returns an unambiguous text for developers, ideally looking like code that rebuilds the object: Vector(3, 4). It is used by the REPL, by repr(), and when an object is inside a list. __str__ returns a readable text for users and is what print uses. If only __repr__ exists, print uses it too, so write __repr__ first.

3. 3

   __eq__ defines ==. By default two objects are equal only if they are the very same object. Return NotImplemented (not False) when the other side is a type you do not understand, so Python can try the other side's method. A hash is a whole number that dictionaries and sets use to find an item fast (Topic 4.5). Equal objects must have equal hashes. So when you define __eq__, Python sets __hash__ to None, and the object becomes unhashable. If you want objects as dictionary keys or set members, define __hash__ too, from the same fields (hash((self.x, self.y))). Then keep those fields unchanged.

4. 4

   For ordering, define __lt__ and use the decorator functools.total_ordering (decorators: Topic 9.6) to get <=, >, and >= from __lt__ and __eq__. Or use dataclasses with order=True (Topic 8.8). Sorting only needs __lt__.

5. 5

   Make an object behave like a collection with __len__ (the len() function), __getitem__ (indexing and slicing), __contains__ (the in operator) and __iter__ (the for loop; Topic 9.1). Python has fallbacks: with only __getitem__, for still works, because Python tries obj[0], obj[1], ... until IndexError. Also, bool(obj) uses __bool__ and falls back to __len__, so an object with length zero is falsy (Topic 1.4).

6. 6

   Operators map to methods: + to __add__, * to __mul__, - to __sub__, abs() to __abs__. If the left side cannot handle the right side's type, Python tries the reflected method on the right side: 5 + v tries int.__add__, fails, then calls v.__radd__(5). That is how sum([v1, v2]) works: sum starts from 0, so __radd__ must accept 0.

7. 7

   __call__ makes an object callable like a function: hello("Asha") runs hello.__call__("Asha"). It is useful for objects that carry configuration or state but are used as functions. A model client or a retry wrapper can be a callable object. Do not overuse special methods: add the ones whose meaning is obvious to every reader.

• ↔

  Special methods make your classes feel built-in, but every one is a promise. If __eq__ or __add__ surprises readers, the clever syntax costs more than a named method would.

• ↔

  Operator overloading is powerful for maths-like types (vectors, money, versions). For anything else a plain method with a clear name (merge, combine) is easier to read and search for.

• Done when you can write __repr__ for any class you create and explain how it differs from __str__.

• Done when you can explain the __eq__ and __hash__ contract.

• Done when you can make a class work with len, indexing, in and for.

• Done when you can explain reflected methods such as __radd__ and NotImplemented.

• Done when you can make an object callable with __call__.