Command Palette

Search for a command to run...

Hectal
Phase 9Intermediate10 of 18 in Python

Iterators, Generators and Decorators

How for really works, generators that produce values one at a time, itertools and functools, decorators, and context managers you write yourself.

Python code is full of loops, but few learners know what a loop does underneath. This phase opens the box: the iteration protocol that every for loop uses, then generators, which let a function hand back values one at a time instead of building a whole list first.

Next come the standard-library toolboxes built on those ideas (itertools and functools), and then decorators: the @ lines that wrap a function with extra behaviour such as logging, caching or retrying. You will write simple ones, ones that take arguments, ones written as classes, and your own context managers with contextlib.

These are the tools behind real AI code. A generator that yields tokens is how a streamed model answer is read. batched splits documents into groups for an embedding API. A retry decorator survives a flaky network call. lru_cache avoids paying twice for the same lookup. The phase ends with functional style, so you can choose well between a comprehension, map and a loop.

0/9 · 0%
9 topics ~4 h 25 code blocks & diagrams
Start with the first topic
1
9.1

How for Really Works: Iterables and Iterators

A for loop asks an object for an iterator with iter(), then calls next() on it again and again until it raises StopIteration. Knowing the difference between an iterable (can be looped over) and an iterator (remembers its place, used up once) explains many surprises.

29 min 1 diagram 2 code practice

2
9.2

Generators and yield

A generator function uses yield to hand back values one at a time and freezes in place between them. It writes an iterator in a few lines, keeps its own state, and can produce infinite streams without using memory.

29 min 1 diagram 3 code practice

3
9.3

Generator Expressions and Lazy Pipelines

A generator expression looks like a list comprehension with round brackets, but it produces values one at a time instead of building a list. Chained together, generator expressions make memory-friendly pipelines, and they pair well with any, all and sum.

26 min 2 code practice

4
9.4

itertools

itertools is the standard library's toolbox for iterators: infinite counters, chaining, slicing, grouping, combinations, running totals, and splitting into batches. Every tool is lazy and works on any iterable.

32 min 1 diagram 3 code practice

5
9.5

functools

functools holds tools that build new functions from old ones: partial fixes some arguments, reduce folds a sequence into one value, lru_cache and cache remember results, cached_property computes an attribute once, singledispatch picks code by type, and wraps keeps a wrapper honest.

27 min 2 code practice

6
9.6

Decorators

A decorator is a function that takes a function and returns a new one, usually a wrapper that adds behaviour before or after the original. The @name line above a def is shorthand for rebinding the name to the wrapped version.

27 min 3 code practice

7
9.7

Decorators with Arguments and Class Decorators

A decorator that takes its own settings, such as @retry(attempts=3), needs three levels of functions: one to take the settings, one to take the function, one to wrap the call. Decorators can also be written as classes, applied to methods, and applied to whole classes.

26 min 2 code practice

8
9.8

Writing Context Managers with contextlib

contextlib lets you write your own with blocks using a generator and @contextmanager, and gives tools for common jobs: suppress to ignore an error, ExitStack to manage a changing number of resources, closing, redirect_stdout and nullcontext.

25 min 3 code practice

9
9.9

Functional Style in Python

Functional style means building programs from small pure functions that take values and return new values, instead of changing things in place. Python mixes it freely with other styles: map, filter, reduce, comprehensions, operator and immutable data are its tools.

26 min 2 code practice

Practise this phase

Hands-on exercises: write the code, run it, and press Check to have it tested automatically.

1

Write a countdown generator

Write a generator function countdown(n) that yields n, then n - 1, and so on down to 1. countdown(0) yields nothing.

Beginner
2

Write an iterator class

Write a class Evens whose constructor takes a limit. Looping over Evens(limit) produces the even numbers from 0 up to but not including limit.

Intermediate
3

Split a stream into batches

Write a generator batches(items, size) that yields lists of up to size items taken from any iterable, in order. The last batch may be shorter. It must work lazily, even on an endless iterable. If size is less than 1, raise ValueError.

Intermediate
4

Write a call-counting decorator

Write a decorator count_calls that counts how many times the decorated function has been called. The count must be readable as function.calls (starting at 0). The decorated function must keep its name and docstring, and pass all arguments through and return the result.

Intermediate
5

Write a retry decorator with settings

Write a decorator factory retry(attempts). The decorated function is called; if it raises ConnectionError it is called again, up to attempts calls in total. If the last attempt also fails, the ConnectionError is raised. Other kinds of error are not retried. The result of a successful call is returned, and arguments are passed through.

Advanced
6

Write a context manager that always cleans up

Using @contextmanager, write logged(name). On entering the with block it prints enter <name>; on leaving it prints exit <name>, even if the block raises an error. The error must still reach the caller. It should yield the name in upper case.

Advanced