Sparse autoencoder
An overcomplete dictionary trained to reconstruct activations from few features.
What it commits you to
A sparse autoencoder decomposes a layer's activations into a few thousand learned features at a time, with a sparsity penalty and a reconstruction loss. If features turn out to be interpretable, a model's internals become auditable rather than merely observable. The bet is monosemanticity; the risk is that a feature which is legible to a human is not causal.
Which realms lean on this
The vocabulary of measurement, and therefore of the disagreements measurement cannot settle.
technical · 1 realm — The Longview Foundry.
See also
Mechanistic interpretability
alignmentReverse-engineering the learned features inside a model rather than inferring them from outputs.
Activation steering
technicalEditing behaviour by adding a direction to the residual stream at inference time.
Interpretability
alignmentThe effort to make what a model represents and computes legible to people.
Model organism
alignmentA small system studied closely enough that its failures teach you something general.