Fact
Advanced
The 'Transformer' Architecture
March 9, 2025
Most modern LLMs, including GPT and Gemini, are based on the Transformer architecture, introduced in the 2017 paper 'Attention Is All You Need.' Its key innovation is the attention mechanism.
Category: LLM Architecture
Difficulty: Advanced