Pipeline: limit, timeout, retry and collect
A fake model with three behaviours. Change the timeout to 0.5 to see 'hang' still fail, or the number of tries to 2 to see 'hard' fail.
Pick the model from the bottleneck: many API calls mean asyncio, a blocking library means threads, CPU-bound Python means processes, and array maths means NumPy. This topic gives the decision guide, a full async pipeline with a limit, timeout and retries, and the bugs that most often go wrong.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write a helper retry(make_coro, tries) that awaits make_coro() up to tries times, retrying only on ConnectionError, and re-raises the last error if all tries fail. Test it with a function that fails once.
Write run_limited(jobs, limit) that takes a list of zero-argument coroutine functions and returns their results in order, running at most limit at once.
Write first_result(coros) that runs several coroutines together and returns the result of whichever finishes first, cancelling the rest (use asyncio.wait with FIRST_COMPLETED).
Explain it without notes
A script calls an LLM API 2,000 times. Describe how you would make it fast and safe.
When would you choose threads, processes or NumPy over asyncio?
What are three bugs that commonly appear in async Python, and how do you detect each?
What is structured concurrency, and which asyncio features support it?
Your async service becomes slow and unresponsive under load, and CPU is at 100% on one core. How do you find the cause and what are the usual fixes?
Expected output
hard: attempt 1 failed (ConnectionError)
hard: attempt 2 failed (ConnectionError)
hang: attempt 1 failed (TimeoutError)
hang: attempt 2 failed (TimeoutError)
hang: attempt 3 failed (TimeoutError)
fast -> answer(fast)
hard -> answer(hard)
hang -> FAILED: hang: gave up after 3 tries