Two regressors in a linear model have correlation 0.95. Compute the variance inflation factor for each coefficient, say by what factor the standard errors rise relative to an orthogonal design, and state what the fit itself loses.

Two regressors in a linear model have correlation 0.95. Compute the variance inflation factor for each coefficient, say by what factor the standard errors rise relative to an orthogonal design, and state what the fit itself loses.

Approach: Regress each regressor on the other to get the auxiliary R-squared, then use the standard variance formula for a single OLS coefficient.

10.26. The variance of a single OLS coefficient is sigma^2/(SST_j*(1 - R_j^2)) where R_j^2 comes from regressing regressor j on the others. Here that auxiliary regression has R_j^2 = 0.95^2 = 0.9025, so the variance inflation factor is 1/(1 - 0.9025) = 10.26 for each of the two coefficients. Standard errors scale with its square root, so they are sqrt(10.26) = 3.20 times what an orthogonal design of the same size would deliver, turning a t-statistic of 6.4 into 2.0. The fitted values, the residuals and R-squared are untouched, since multicollinearity only affects how the shared variation is split between the two coefficients and not how much of y the pair explains together. An F-test of the two jointly remains valid and will often reject when neither individual t-statistic does, which is the diagnostic signature.

Follow-up: You replace the two regressors with their first principal component. What is gained, and what interpretation is lost?

Key concepts: variance inflation factor, multicollinearity, standard error, orthogonal design.