What is 'Information Bottleneck'?

What is 'Information Bottleneck'?

Approach: Describe a model's goal to compress input while preserving information about the output.

A theoretical framework stating that a neural network aims to squeeze out all information about the input X that is not relevant to predicting the output Y, retaining only the most relevant features.

Follow-up: How does it relate to the generalization capabilities of deep networks?

Key concepts: mutual information, compression, relevance.