Async IO & Concurrency

Core concepts

1. async def / await: Coroutines scheduled on an event loop. — Python docs: Coroutines and tasks
2. asyncio.gather: Run multiple I/O tasks concurrently. — Python docs: asyncio.gather
3. Blocking code: Never call time.sleep or sync HTTP inside async routes — use asyncio.to_thread or async libraries. — Python docs: Running in threads
4. Agent loops: Tool-calling agents are naturally async — see Python AI SDK comparison. — asyncio documentation

import asyncio
import httpx

async def fetch_title(client: httpx.AsyncClient, url: str) -> str:
    r = await client.get(url, timeout=10)
    r.raise_for_status()
    return url

async def main():
    async with httpx.AsyncClient() as client:
        urls = ["https://example.com", "https://ihatereading.in"]
        results = await asyncio.gather(*(fetch_title(client, u) for u in urls))
        print(results)

asyncio.run(main())

1. uvloop (optional): Faster event loop on Linux/macOS for high-throughput services. — uvloop docs

Concept videos

YouTube lessons for these topics. On a large screen they open in the reading panel next to the roadmap, same as the article links on each core concept.

• asyncio — Python async foundations (Corey Schafer — start here)

Explanation

Async Python is a single waiter juggling many tables — not many chefs. While one coroutine waits on network I/O, others progress. CPU-heavy cooking still needs separate processes.

async def / await

async def defines a coroutine function; calling it returns a coroutine object, not a result. await yields control until I/O completes. Think of await as “pause this function, let others run, resume when data arrives.”

asyncio.gather

gather schedules multiple coroutines and collects results. Order matches input order; one failure can cancel siblings unless you use return_exceptions=True. Perfect for fan-out HTTP calls in agents or dashboards.

Blocking code

Sleeping or synchronous HTTP inside async code freezes the waiter — every table stops. Use await asyncio.sleep, async HTTP clients (httpx, aiohttp), or offload blocking work with asyncio.to_thread / executors.

Agent loops

LLM tool calls spend time waiting on APIs — async fits naturally. Agent frameworks assume you understand await boundaries so tools can run concurrently without blocking the UI or server.

uvloop

uvloop replaces the default loop with a faster implementation (libuv). It is an optimization, not a prerequisite — measure before adopting.

What Is It?

asyncio provides an event loop, coroutines (async def), and awaitable I/O libraries.

concurrent.futures offers thread and process pools for blocking or CPU work.

The GIL limits parallel CPU execution in threads but I/O releases it often enough for many network servers.

Why Should I Care?

Agent and API code spends most time waiting on models and HTTP — async keeps latency down under concurrency.

Mixing blocking calls inside async functions stalls the entire loop.

Real World Analogy

asyncio is a single waiter juggling many tables: they take orders, kick off kitchen timers, and serve other tables instead of staring at one oven door.

Visual Mental Model

event loop
  |-- task A await I/O
  |-- task B await I/O
  (one thread, cooperative)

Beginner Explanation

Use async for I/O-bound work (HTTP, DB, LLM calls). CPU-bound work belongs in processes or native code, not tight async loops.

Smallest Possible Example

import asyncio

async def main():
    await asyncio.sleep(0.1)
    print("done")

asyncio.run(main())

Common Mistakes

❌ Calling time.sleep inside async code.

Blocks the loop. Use await asyncio.sleep instead.

❌ Creating unbounded tasks without backpressure.

You exhaust memory and file descriptors. Use semaphores or queues.

❌ Using threads for CPU-heavy math on CPython.

Processes or vectorized libraries bypass GIL contention.

Real Project Example

FastAPI endpoints are often async and call async HTTP clients to microservices.

httpx AsyncClient powers many LLM gateway integrations.

Build This Mini Project

Fetch three public HTTP endpoints concurrently with asyncio.gather and compare wall time to sequential awaits.

Interview Question

When should you use asyncio vs threading?

Asyncio excels at many concurrent I/O-bound tasks with explicit await points. Threads help when libraries are blocking and I/O-bound but not async-aware. CPU-bound parallelism usually means multiprocessing.

Key Takeaways

• Async for I/O, not heavy CPU.
• Never block the event loop.
• Understand GIL limits on threads.
• Use gather/semaphores for concurrency control.

Resources

• asyncio documentation
• Real Python: Async IO

The 60-second story

You are on a phone call with a supplier who put you on hold. A bad waiter stands there staring at the hold music and ignores every other table. A good waiter covers the mouthpiece, refills water, and comes back when the supplier speaks. async and await are the good waiter. The event loop is the floor manager who notices the supplier is back.

async def writes a task that is allowed to pause. await is the pause, not a nap of the whole restaurant. asyncio.gather is sending three waiters to three holds and plating whatever comes back. It is still one kitchen process unless you explicitly hire more. Waiting overlaps. Heavy chopping does not, unless you hand the knife to someone else.

time.sleep inside an async route is the staring waiter. The floor manager cannot cut in, because you froze the only person who was listening for “they’re back.” Use an async sleep, or asyncio.to_thread when a library only knows how to stare. Sync HTTP in an async function is the same mistake with a network cable.

Agent loops feel like this all day: ask a model, call a tool, wait, ask again. If each wait freezes the process, one user blocks the room.

Map the floor.

• async def is a task that knows how to pause.
• await is covering the phone, not locking the door.
• The event loop is the floor manager.
• gather is several holds at once.
• time.sleep in async code is staring.
• asyncio.to_thread is handing a blocking job to another worker.
• uvloop is a faster floor manager on some machines, not a second restaurant.

import asyncio

async def fetch_one(name):
    await asyncio.sleep(1)
    return name

async def main():
    return await asyncio.gather(fetch_one("a"), fetch_one("b"))

Both names come back in about a second, not two, because the sleeps overlap. That demo is the whole idea. If the body of fetch_one did a huge CPU loop, they would not overlap. The waiter would be chopping, not waiting.

Async was added because servers spend their lives waiting on networks and databases. Threads are extra waiters, with extra ways to grab the same plate. For I/O, one well-run floor is simpler. For CPU, you still need another process or a thread pool. The GIL, on the interview page, is why extra threads do not magically chop faster in standard Python.

Mental model: do not stare at hold music. Await means the floor stays open. Blocking calls smuggle the stare back in.

You can explain it if you say: gather overlaps waits, sleep and sync HTTP do not belong in the async path, and async is not the same thing as parallel CPU.