A strategy allocates one vector of 200 doubles per market data update at 400k updates a second. The median latency is 3 microseconds and the 99.9th percentile is 40 microseconds. Say what you suspect, how you prove it, and three fixes ranked by how much they change the code.

A strategy allocates one vector of 200 doubles per market data update at 400k updates a second. The median latency is 3 microseconds and the 99.9th percentile is 40 microseconds. Say what you suspect, how you prove it, and three fixes ranked by how much they change the code.

Approach: Put a number on a malloc call in the common case and in the bad case, then ask what makes the bad case rare and correlated with the latency tail.

Suspect the per-update heap allocation, prove it with a profile that samples the malloc and page fault paths plus a histogram of allocation duration, and fix it by reusing one preallocated vector across updates, then a thread local arena, then a fixed size stack buffer with a static assert on the 200. A malloc served from a warm thread cache is 20 to 50 nanoseconds, which is why the median is untouched, and the tail comes from the rare path where the allocator takes a lock, asks the kernel for memory and touches fresh pages, each of which is a page fault costing several microseconds. Reusing one vector and calling clear rather than destroying it keeps the capacity and removes the allocation entirely, which is a two line change. The arena helps once the sizes vary. The stack buffer is fastest and fixes the size into the code.

Follow-up: After the allocation is gone the same spike still appears once a minute. What do you look at next?

Key concepts: heap allocation, tail latency, page fault, preallocation.