Trivia
Advanced
BERT's Contribution
August 29, 2025
Q
What major innovation did the BERT model (Bidirectional Encoder Representations from Transformers) introduce?
A
BERT was the first deeply bidirectional, unsupervised language representation, pre-trained from only a plain text corpus. Unlike previous models that were unidirectional, BERT could understand the context of a word based on both what came before and after it.
Category: LLM Architecture
Difficulty: Advanced