Replacing a raw function pointer callback with std::function on a path that fires 500k times a second added 40 nanoseconds a call and put malloc in the profile. The lambda captures three pointers and a 24 byte struct. Explain the mechanism and give a fix that keeps the same call site.

Replacing a raw function pointer callback with std::function on a path that fires 500k times a second added 40 nanoseconds a call and put malloc in the profile. The lambda captures three pointers and a 24 byte struct. Explain the mechanism and give a fix that keeps the same call site.

Approach: Work out how a type-erasing wrapper stores a callable whose size it does not know, then compare the size of the capture against the space the wrapper holds inline.

The capture is larger than the inline space in std::function, so each construction is a heap allocation and each invocation pays an indirect call through the type erasure boundary, and the fix is to shrink the capture below the inline size or to template the caller on the callable type. Implementations reserve roughly 16 to 32 bytes inline, and three pointers plus a 24 byte struct is about 48 bytes, so the small buffer optimisation does not apply and every copy of the callable is a malloc and a free. The indirect call also blocks inlining of the callback body, which is usually the larger half of the 40 nanoseconds. Capturing a pointer to a stable context object instead of the struct brings the capture to one pointer, which fits inline and removes the allocation, and a template parameter at the call site removes the indirect call as well.

Follow-up: How would you confirm the allocations come from that call site rather than elsewhere, on a process already running in production?

Key concepts: small buffer optimisation, type erasure, heap allocation, indirect call.