A Gaussian process with a squared exponential kernel is fitted to a volatility surface and extrapolates to a flat mean far from the data. State the posterior mean and variance formulas, explain that behaviour, and say what changes with a Matern kernel of smoothness 3/2.

A Gaussian process with a squared exponential kernel is fitted to a volatility surface and extrapolates to a flat mean far from the data. State the posterior mean and variance formulas, explain that behaviour, and say what changes with a Matern kernel of smoothness 3/2.

Approach: Write the conditional of a joint Gaussian, then examine what happens to the cross-covariance vector as the test point moves away from every training point.

The posterior is Gaussian with mean k_*^T (K + sigma^2 I)^{-1} y and variance k_** - k_*^T (K + sigma^2 I)^{-1} k_*, so far from the data the cross-covariance vector k_* goes to zero and the mean reverts to the prior mean while the variance rises to the prior variance k_**. That reversion is a property of the kernel: the squared exponential decays like exp(-r^2/(2*l^2)), which falls faster than any polynomial, so at three or four length scales the correlation with the observed points is numerically zero and the model has nothing to extrapolate with. The kernel is the entire model specification, since the mean function is usually taken as zero and every assumption about smoothness, periodicity and length scale lives in k. The squared exponential also implies sample paths that are infinitely differentiable, which is a strong and usually false assumption for a market surface and produces an overconfident interpolation between distant quotes. A Matern kernel with smoothness 3/2 gives paths that are once differentiable, decays more slowly at moderate distance, and therefore fits rougher structure and widens the uncertainty between observations in a way that better matches quoted data. The noise term sigma^2 on the diagonal is what keeps the linear system invertible and controls how closely the mean passes through each observation.

Follow-up: The exact posterior costs O(n^3) to fit. What two approximations reduce that, and what does each assume about the structure of the kernel matrix?

Key concepts: posterior mean, posterior variance, kernel, length scale.