A feed handler writes to an unbounded queue read by a slower strategy thread. Describe what happens over a 30-second burst where the producer runs 5% faster than the consumer at 200,000 messages per…

Question

A feed handler writes to an unbounded queue read by a slower strategy thread. Describe what happens over a 30-second burst where the producer runs 5% faster than the consumer at 200,000 messages per second, including memory and latency, and give the two bounded designs and what each one throws away.

Accepted Answer

The queue reaches 300,000 messages, the strategy is acting on data 30 seconds old, and the box eventually runs out of memory. A 5% deficit at 200,000/s is 10,000 messages per second accumulating, so 300,000 after 30 seconds. At 128 bytes each that is only 38 MB, so memory is not the first thing to fail; the queueing delay is. By Little's law the delay is backlog divided by service rate, 300,000/190,000 which is about 1.6 seconds and growing linearly, and if the burst continues the delay grows without bound. A strategy quoting on 1.6-second-old prices is not slow, it is wrong, and it will keep sending orders that the market has already moved past. This is bufferbloat in a trading system. The two bounded designs are to drop and to block. A bounded ring that overwrites the oldest entry keeps the consumer on fresh data and throws away history, which is correct for a price feed where only the current book matters. Blocking the producer applies backpressure upstream and throws away nothing, which is correct for an order flow where every message has to be processed, and it forces the loss to happen at the network where a sequence gap makes it visible.