What is the difference between threading, multiprocessing and asyncio in Python?
Quick answer
Threading runs concurrent threads that share memory and suit blocking I/O, multiprocessing runs separate processes to use multiple CPU cores for CPU-bound work, and asyncio runs many non-blocking tasks cooperatively on one thread for high-volume I/O.
Choose by the type of bottleneck. For CPU-bound work such as image processing or number crunching, use multiprocessing so each worker has its own GIL, and accept the cost of starting processes and passing data between them. For I/O-bound work with libraries that block, such as requests, threads are simple and effective.
For thousands of concurrent network connections, asyncio with async/await has the lowest overhead because tasks yield control at each await, but every library in the path must be async-aware, and one blocking call stalls the whole event loop. Many production systems combine them, for example an async web server that sends CPU-heavy jobs to a process pool or task queue.
import asyncio, httpx
async def fetch(client, url):
r = await client.get(url)
return r.status_code
async def main(urls):
async with httpx.AsyncClient() as client:
return await asyncio.gather(*(fetch(client, u) for u in urls))
asyncio.run(main(["https://example.com"] * 5))Key points
- CPU-bound: multiprocessing
- Blocking I/O: threading
- Massive concurrent I/O: asyncio