Activation steering
Editing behaviour by adding a direction to the residual stream at inference time.
What it commits you to
Steering locates a direction in activation space that correlates with a behaviour — the refusal direction is the famous one — and adds it to shift behaviour without retraining. It is cheap, immediate, and reversible, which is why it is attractive to deployers. It is also a cheap jailbreak, since the same edit works on whoever holds it.
Which realms lean on this
The vocabulary of measurement, and therefore of the disagreements measurement cannot settle.
technical · 1 realm — The Longview Foundry.
Canon that discusses this
See also
Sparse autoencoder
technicalAn overcomplete dictionary trained to reconstruct activations from few features.
Mechanistic interpretability
alignmentReverse-engineering the learned features inside a model rather than inferring them from outputs.
Developer mode
alignmentA jailbreak that persuades a model it is talking to its own developers and can drop restrictions.
Red teaming
technicalSystematically attacking a model before users do, to find its failure modes.