Kernel bypass is described as removing the kernel from the data path. List precisely what is removed, what is still there that people forget about, and give the situation where a bypass stack is slower than a tuned socket.

Kernel bypass is described as removing the kernel from the data path. List precisely what is removed, what is still there that people forget about, and give the situation where a bypass stack is slower than a tuned socket.

Approach: Walk the packet from the wire to the application twice, once through the kernel and once through a user space driver, and note each step that disappears and each that does not.

Bypass removes the interrupt, the softirq, the kernel protocol processing, the copy into a socket buffer and the scheduler wakeup, leaving DMA into pinned user memory that the application busy polls. What remains is the wire time, the NIC's own store-and-forward or cut-through delay, the PCIe transfer of a few hundred nanoseconds, the DMA write to memory and the cache miss the application takes reading a descriptor the NIC just wrote. Page pinning and the IOMMU are still involved, and the protocol work has not vanished, it has moved into the user space stack where you now own TCP state, checksums, fragmentation and ARP. A bypass stack is slower when the core spends its poll budget on many idle queues, because polling burns the cycles the strategy needed, and when message rates are low enough that the socket path never actually blocks so its wakeup cost is not paid. It is also slower in throughput terms if the user space stack cannot use segmentation offload or receive-side scaling as well as the kernel does. The honest summary is that bypass buys a low and stable tail at the price of a dedicated core per polled queue.

Follow-up: How does an onload-style library that intercepts the socket API differ from a full user space stack, and which one do you pick for a strategy with 200 TCP sessions?

Key concepts: kernel bypass, dma, busy polling, page pinning.