Rank these by order of magnitude and give a number for each: an L1 cache hit, an L3 hit, a main memory read on the local NUMA node, a read from the remote NUMA node, a switch hop inside a rack, and a round trip New York to London. Then say how many L1 hits fit inside the transatlantic round trip.

Rank these by order of magnitude and give a number for each: an L1 cache hit, an L3 hit, a main memory read on the local NUMA node, a read from the remote NUMA node, a switch hop inside a rack, and a round trip New York to London. Then say how many L1 hits fit inside the transatlantic round trip.

Approach: Work in nanoseconds throughout, quote each level of the hierarchy, then divide the largest by the smallest.

Roughly 1 ns, 40 ns, 80 ns, 140 ns, 500 ns, and 60 ms, so about 60 million L1 hits fit in the London round trip. An L1 hit is about 1 ns, or 4 cycles at 4 GHz. L2 is around 12 ns and L3 around 40 ns because it is shared and further away. A local DRAM read is 80 to 100 ns; crossing the interconnect to the remote NUMA node adds 50 to 60 ns, so 140 ns. One cut-through switch hop plus a short cable is a few hundred ns, so 500 ns for a rack traversal with a NIC on each end. New York to London is about 5500 km, and fibre carries light at roughly 200,000 km/s, giving 27 ms one way and 55 to 60 ms round trip on a real route. Dividing 60 ms by 1 ns gives 6 * 10^7. The lesson for a latency budget is that microarchitecture only matters once the network path is already the shortest available, since no amount of cache tuning recovers a millisecond of propagation delay.

Follow-up: Microwave over the same path runs at about 0.99c through air. How much round trip time does that save and why does anyone still use fibre?

Key concepts: cache hierarchy, numa, propagation delay, latency budget.