What is the Bias-Variance tradeoff?

What is the Bias-Variance tradeoff?

Approach: Explain the tension between error from erroneous assumptions and error from sensitivity to fluctuations.

Bias is error from incorrect assumptions in the learning algorithm (underfitting). Variance is error from sensitivity to small fluctuations in the training set (overfitting). As model complexity increases, bias decreases and variance increases. The goal is to minimize the total error.

Follow-up: How does regularization affect bias and variance?

Key concepts: generalization error, underfitting, overfitting.