What is 'Transfer Learning' in NLP?

What is 'Transfer Learning' in NLP?

Approach: Describe fine-tuning a pre-trained model (like BERT) on a specific task.

The practice of training a large model on a massive dataset (pre-training) and then adapting it to a smaller, specific dataset (fine-tuning). This allows the model to leverage general language knowledge.

Follow-up: Name a famous pre-trained model for NLP. (BERT, GPT)

Key concepts: pre-training, fine-tuning, contextual embeddings.