HardConcept

What is the difference between threading, multiprocessing and asyncio in Python?

Technology:
Python
Experience:
Senior

Quick answer

Threading runs concurrent threads that share memory and suit blocking I/O, multiprocessing runs separate processes to use multiple CPU cores for CPU-bound work, and asyncio runs many non-blocking tasks cooperatively on one thread for high-volume I/O.

Detailed explanation

Choose by the type of bottleneck. For CPU-bound work such as image processing or number crunching, use multiprocessing so each worker has its own GIL, and accept the cost of starting processes and passing data between them. For I/O-bound work with libraries that block, such as requests, threads are simple and effective.

For thousands of concurrent network connections, asyncio with async/await has the lowest overhead because tasks yield control at each await, but every library in the path must be async-aware, and one blocking call stalls the whole event loop. Many production systems combine them, for example an async web server that sends CPU-heavy jobs to a process pool or task queue.

Example

python
import asyncio, httpx

async def fetch(client, url):
    r = await client.get(url)
    return r.status_code

async def main(urls):
    async with httpx.AsyncClient() as client:
        return await asyncio.gather(*(fetch(client, u) for u in urls))

asyncio.run(main(["https://example.com"] * 5))

Key points

  • CPU-bound: multiprocessing
  • Blocking I/O: threading
  • Massive concurrent I/O: asyncio
  • PythonMedium

    What are decorators in Python and how do they work?

    A decorator is a function that takes another function and returns a modified version of it, letting you add behaviour such as logging, caching or access control without changing the original code.

    Mid-level 路 Senior 路 Functions & Scope
  • PythonMedium

    What are generators and the yield keyword in Python?

    A generator is a function that uses yield to produce values one at a time and remembers its state between calls, so it can process large or infinite sequences without holding everything in memory.

    Mid-level 路 Senior 路 Functions & Scope
  • PythonMedium

    How does Python manage memory and garbage collection?

    CPython frees objects mainly through reference counting, deleting an object as soon as nothing refers to it, and uses a cyclic garbage collector to clean up groups of objects that reference each other.

    Mid-level 路 Senior 路 Memory Management
  • PythonEasy

    What is the difference between a list and a tuple in Python?

    A list is mutable and can be changed after creation, while a tuple is immutable; tuples are slightly faster, use less memory and can be used as dictionary keys when their items are hashable.

    Intern 路 Junior 路 Data Structures