A 200 by 50 returns matrix has singular values 10, 6, 3, 1 and the rest zero. Give the error of the best rank 2 approximation in both the Frobenius norm and the spectral norm, and state the fraction of total variance the first two components explain.

A 200 by 50 returns matrix has singular values 10, 6, 3, 1 and the rest zero. Give the error of the best rank 2 approximation in both the Frobenius norm and the spectral norm, and state the fraction of total variance the first two components explain.

Approach: Apply the Eckart-Young theorem: truncating the singular value decomposition after k terms is optimal in both norms, and the residual is built only from the discarded singular values.

sqrt(10). By the Eckart-Young theorem the truncated singular value decomposition is the best rank 2 approximation, and the Frobenius norm error is sqrt(sigma_3^2 + sigma_4^2) = sqrt(9 + 1) = sqrt(10) = 3.162. The spectral norm error is the largest discarded singular value, sigma_3 = 3. Total squared mass is 100 + 36 + 9 + 1 = 146, and the first two carry 136, so the two components explain 136/146 = 93.2% of the variance. The gap between 3 and 3.162 shows the Frobenius norm charges for every discarded direction while the spectral norm charges only for the worst one.

Follow-up: If the singular values were 10, 6, 5.9, 5.8, why would a rank 2 truncation be numerically unstable even though the error is still computable?

Key concepts: eckart-young theorem, singular value decomposition, frobenius norm, spectral norm.