State exactly what batch normalisation computes at training time and what it computes at inference time. Explain why the two differ, and what goes wrong if a model is deployed with batch size 1 while still using batch statistics.

State exactly what batch normalisation computes at training time and what it computes at inference time. Explain why the two differ, and what goes wrong if a model is deployed with batch size 1 while still using batch statistics.

Approach: Write the normalisation with the minibatch mean and variance, then ask what statistics are available when a single example is scored.

At training time batch normalisation computes (x - mu_B)/sqrt(var_B + epsilon) * gamma + beta using the current minibatch mean and variance, and at inference it substitutes fixed running estimates of the mean and variance collected during training, so the layer becomes a deterministic affine map that can be folded into the preceding weights. The two must differ because a prediction may not depend on which other examples happen to be scored alongside it, and at training the batch statistics also act as a noise source that regularises. Deploying with batch statistics at batch size 1 gives var_B = 0 for every feature, so the normalised output is zero up to epsilon and every input maps to beta, destroying the model. With a small live batch the same effect appears as a high-variance estimate of mu and var, so the same input produces different predictions depending on its neighbours in the batch, which on a trading system means the signal for one instrument depends on which other instruments were requested. The running estimates must also be collected over data representative of live inputs, so a model trained on a shifted distribution carries stale mu and var into production and normalises against the wrong centre. Layer normalisation avoids the issue entirely by normalising across features within one example.

Follow-up: How do you fold a batch normalisation layer into the preceding linear layer at inference, and what are the resulting weight and bias?

Key concepts: batch normalization, running statistics, inference mode, batch dependence.