Transformer
The neural network architecture behind nearly every current language model, built around attention rather than sequential processing.
Its key property is that every position in a sequence can look at every earlier position in one step, and all positions can be computed in parallel. That parallelism, more than any conceptual breakthrough, is what made training at the current scale affordable.
Where this comes up
Articles that define transformer in context.
What a transformer is actually doing when it "reads"
Attention, explained through a single nine-token sentence: what the weights mean, why the upper triangle of the matrix is empty, and what that mask has to do with the model's failure modes.
What is Artificial Intelligence?
AI, machine learning, deep learning and generative models are nested, not synonymous. Here is what distinguishes each layer, and which one people now mean when they say AI.