Pick between asyncio, threads and processes for three jobs in one Python service: 2000 concurrent REST calls to venues, a 40 second pure Python backtest loop over 10 million rows, and 300 MB a second of protobuf decoding in a C extension that releases the interpreter lock. Justify each choice.

Pick between asyncio, threads and processes for three jobs in one Python service: 2000 concurrent REST calls to venues, a 40 second pure Python backtest loop over 10 million rows, and 300 MB a second of protobuf decoding in a C extension that releases the interpreter lock. Justify each choice.

Approach: For each job decide whether the time goes on waiting for a descriptor, running bytecode, or sitting inside a C call, and match that to what each concurrency model is able to overlap.

Asyncio for the REST calls, processes for the backtest loop, threads for the protobuf decoding. The REST calls wait on sockets, so a single asyncio event loop overlaps all 2000 at about a kilobyte of state each, where 2000 OS threads would cost gigabytes of stacks and heavy scheduling. The backtest is pure Python bytecode, so the global interpreter lock serialises threads and only process parallelism helps, at the price of pickling the inputs and having no shared state, which suits a loop splittable by date. The protobuf decoding runs inside a C extension that drops the lock, so threads decode in parallel with no serialisation and without moving 300 MB a second through a pipe as processes would have to. In one service that means the event loop on the main thread and the other two dispatched to a process pool and a thread pool.

Follow-up: What happens when a blocking call is added inside a coroutine on the event loop, and how would you detect it in production?

Key concepts: asyncio event loop, global interpreter lock, process parallelism, c extension.