A Java strategy places its first order of the day 400 microseconds slower than its steady-state figure, and a C++ strategy on the same box shows the same pattern at 90 microseconds. Give every…

Question

A Java strategy places its first order of the day 400 microseconds slower than its steady-state figure, and a C++ strategy on the same box shows the same pattern at 90 microseconds. Give every distinct cause of a cold first message and the mitigation for each.

Accepted Answer

The 90 microseconds common to both is cold caches, cold TLB, page faults and cold branch predictors, and the extra 310 in Java is interpreted bytecode before JIT compilation plus class loading. Every process starts with an empty instruction cache and data cache, so the first pass through the hot path takes a DRAM miss at nearly every step. Pages are mapped lazily, so the first touch of the order buffer, the log buffer and the stack takes a minor page fault costing a few microseconds each, and if the binary's text pages are not resident the first call takes a major fault. Branch predictors and the indirect branch target buffer have no history so the first pass mispredicts repeatedly. In Java the method runs interpreted until the invocation counter crosses the compilation threshold, and the first allocation may trigger a young collection. The mitigations are the same in both languages: send warm-up traffic through the real code path before the open, on a shadow session or with a flag that suppresses the send at the last step, pre-fault and lock memory with mlockall, pre-touch every buffer, and in Java run enough warm-up iterations to force tiered compilation to the top tier and to reach a stable heap.