An order handler calls a virtual on_tick through a base pointer once per market data message at 2 million messages a second. Quantify what the virtual call costs beyond a direct call, then give three ways to remove it and say what each one gives up.

An order handler calls a virtual on_tick through a base pointer once per market data message at 2 million messages a second. Quantify what the virtual call costs beyond a direct call, then give three ways to remove it and say what each one gives up.

Approach: Break the call into the extra load, the indirect branch and the lost inlining, then consider compile-time alternatives that fix the target at each call site.

The direct cost is one extra load of the vtable pointer plus an indirect branch, roughly 1 to 3 nanoseconds when both hit cache, and the larger cost is the lost inlining and the branch target buffer misses once the call site sees several types, which together reach 10 to 20 nanoseconds a message. A CRTP base templated on the derived type fixes the type at compile time and gives up loading a strategy chosen at runtime. A variant of the concrete types visited at the call site keeps one binary and gives up open extension by third party code. Marking the class or the method final lets the compiler prove the target, which is devirtualisation, and gives up further derivation. At 2 million messages a second, saving 10 nanoseconds returns 20 milliseconds of CPU per second, which is 2 percent of a core.

Follow-up: How would you measure the branch target buffer miss rate at that one call site on a running process?

Key concepts: vtable, indirect branch, inlining, crtp, devirtualisation.