What is 'Weight Initialization' and why is it important?

What is 'Weight Initialization' and why is it important?

Approach: Explain the starting values of neural network weights.

The process of setting the initial values of weights in a neural network. Proper initialization is crucial to prevent vanishing or exploding gradients and to ensure the network breaks symmetry during training.

Follow-up: What happens if all weights are initialized to zero?

Key concepts: symmetry breaking, xavier, he.