Transformer
The attention-based architecture underlying essentially all current large models.
What it commits you to
Transformers replaced recurrence with attention, which made training parallel at scale and is the reason modern capability tracks compute so directly. Their consequence for alignment is structural rather than incidental: the weights are the model, there is no modular decomposition to audit, and the residual stream can be edited in ways that change behaviour without retraining. Every interpretability problem of the field is a problem about this architecture.
Which realms lean on this
The vocabulary of measurement, and therefore of the disagreements measurement cannot settle.
technical · 1 realm — The Longview Foundry.
See also
Scaling law
technicalPredictable power-law relationships between model size, compute and loss.
Mechanistic interpretability
alignmentReverse-engineering the learned features inside a model rather than inferring them from outputs.
Emergent capability
technicalA capability that appears abruptly above a scale threshold rather than gradually.
Reinforcement learning
technicalLearning a policy by optimising cumulative reward against an environment.