Concurrency in Python: threads, processes and asyncio compared

Sooner or later every programmer faces a program that is too slow because it does one thing at a time: it downloads a hundred pages one after another, processes thousands of images in sequence, waits for a service's response before moving on to the next request. The solution is to let several activities progress together. But "together" can mean different things, and choosing the wrong tool may bring no benefit at all or, worse, introduce errors that are hard to reproduce.

Concurrency and parallelism

Two terms that are often confused:

• concurrency: managing several activities that progress over the same period, taking turns. A waiter serving ten tables is concurrent: they do not serve two tables at the same instant, but they do not wait for the first customer to finish eating before taking the second one's order;
• parallelism: running several activities at the same instant, on several processing units. Ten waiters serving ten tables work in parallel.

Concurrency is about the structure of the program; parallelism is about physical execution. You can have concurrency without parallelism, on a single core, and the benefits can still be enormous.

The decisive question: where is time being lost?

Before choosing a tool, you need to understand what kind of work is slowing the program down.

• I/O-bound: the program spends most of its time waiting. Network requests, database queries, file reads. The CPU is almost idle.
• CPU-bound: the program spends its time computing. Image processing, compression, numerical calculations, analysing large amounts of text.

I/O-bound work needs concurrency: while waiting for one response, you start another request. CPU-bound work needs parallelism: several cores computing at once.

Python's GIL

In the standard implementation of Python, CPython, there is the Global Interpreter Lock: a mechanism that allows only one thread at a time to execute Python code. The practical consequence is that threads do not speed up pure computational Python code. During I/O operations, however, the lock is released, so threads work well for waiting-heavy work.

Recent versions of Python have introduced an experimental variant without the GIL, but it is not the default at the moment: for most projects the rule just described still holds.

Three tools

Threads

Threads share the process's memory and suit I/O-bound work with traditional synchronous libraries. The simplest way to use them is a pool:

from concurrent.futures import ThreadPoolExecutor
import urllib.request

def download(url):
    with urllib.request.urlopen(url, timeout=10) as r:
        return len(r.read())

with ThreadPoolExecutor(max_workers=10) as pool:
    sizes = list(pool.map(download, url_list))

Ten downloads proceed together instead of in sequence.

Processes

Each process has its own interpreter and memory, so the GIL is not a limit: several processes really do use several cores. They suit CPU-bound work.

from concurrent.futures import ProcessPoolExecutor

with ProcessPoolExecutor() as pool:
    results = list(pool.map(process_image, paths))

The price is a higher startup cost and the need to transfer data between processes by serialising it. It is therefore worthwhile for tasks heavy enough to justify that cost.

asyncio

asyncio lets you write concurrent code in a single thread, using async and await. Each await marks a point where the function can suspend and let the others work while it waits.

import asyncio

async def check(service):
    await asyncio.sleep(1)    # simulates a network call
    return f"{service}: ok"

async def main():
    results = await asyncio.gather(*(check(s) for s in services))

It is very efficient with thousands of simultaneous connections, but it requires compatible libraries: a slow synchronous call inside an async function blocks the entire event loop.

How to choose

The dangers of shared memory

With threads, several activities can modify the same data. An apparently simple operation like counter += 1 is not atomic: two threads can read the same value and write the same result, losing an increment. These errors, called race conditions, are intermittent and hard to reproduce. The main defences:

1. avoid sharing mutable data, by having each task return its own result, as in the pools shown above;
2. use structures designed for concurrency, such as queue.Queue;
3. protect critical sections with a threading.Lock when sharing is unavoidable.

In short

• Concurrency interleaves several activities; parallelism runs them at the same instant on several cores.
• Before choosing, work out whether the program is limited by waiting (I/O) or by computation (CPU).
• In CPython the GIL stops threads from speeding up pure computation, but not I/O.
• Threads and asyncio for waiting, processes for intensive computation.
• Avoid sharing mutable data between threads: it is the source of race conditions.