What is 'Over-Parameterization'?

What is 'Over-Parameterization'?

Approach: Describe models with far more parameters than training samples.

A regime in deep learning where a model has many more parameters than the number of training data points. Surprisingly, these models can generalize well despite having the capacity to overfit.

Follow-up: What is the 'Double Descent' curve?

Key concepts: double descent, capacity, regularization.