Orders arrive at a risk checker at 50,000 per second and each check takes 18 microseconds on one thread. Compute the utilisation, then use an M/M/1 approximation to give the mean queue length and mean total time in system. Then recompute at 54,000 per second and state what the extra 8% of load did.

Orders arrive at a risk checker at 50,000 per second and each check takes 18 microseconds on one thread. Compute the utilisation, then use an M/M/1 approximation to give the mean queue length and mean total time in system. Then recompute at 54,000 per second and state what the extra 8% of load did.

Approach: Convert the service time to a service rate, form rho, apply the standard M/M/1 expressions for mean number in system and mean sojourn time, then repeat at the higher arrival rate.

At 50,000/s utilisation is 0.90, mean number in system is 9 and mean time is 180 microseconds; at 54,000/s utilisation is 0.972, mean number is 34.7 and mean time is 643 microseconds. Service rate mu is 1/18 microseconds, which is 55,556 per second. At lambda = 50,000, rho = 50,000/55,556 = 0.90. M/M/1 gives L = rho/(1-rho) = 0.9/0.1 = 9 and W = 1/(mu - lambda) = 1/5,556 s = 180 us. Little's law checks it, since lambda * W is 50,000 * 180e-6, which is 9. At lambda = 54,000, rho is 0.972, L is 0.972/0.028, which is 34.7, and W is 1/1,556 s, which is 643 us. An 8% increase in load multiplied queueing time by 3.6, because W scales as 1/(1-rho) and the reciprocal blows up near one. This is the reason capacity planning for a trading path targets utilisation around 0.3 to 0.5 rather than 0.9: the headroom is not waste, it is what keeps the tail bounded when a burst arrives.

Follow-up: Real order arrivals are bursty rather than Poisson. Does that make the M/M/1 answer optimistic or pessimistic, and which formula do you move to?

Key concepts: utilisation, little's law, queueing, service rate.