Interpretability
The effort to make what a model represents and computes legible to people.
What it commits you to
Interpretability has two incompatible goals: making the model debuggable, and making the model safe. Saliency methods served the first. Neither serves the second, because a legible component is not a legible system. The field's honest position in 2026 is that we have instruments and no theory of what would constitute sufficient evidence.
Which realms lean on this
Every realm has to keep a system honest. These are the terms that say out loud what the other kinds keep implicit.
alignment · 5 realms — The Benthamite Foundry, The Existential Wilds, The Longview Foundry, The Stoic Ramparts, The Ubuntu Weave.
See also
Mechanistic interpretability
alignmentReverse-engineering the learned features inside a model rather than inferring them from outputs.
Sparse autoencoder
technicalAn overcomplete dictionary trained to reconstruct activations from few features.
Activation steering
technicalEditing behaviour by adding a direction to the residual stream at inference time.
Model organism
alignmentA small system studied closely enough that its failures teach you something general.