Red teaming
Systematically attacking a model before users do, to find its failure modes.
What it commits you to
Red teaming applies adversarial pressure deliberately and in advance: automated attackers generate prompts, automated or human judges classify the damage, and the findings feed both training and deployment review. It works well enough to be standard and poorly enough to be defeatable, because the attacker has the same access to the defender's publications as anyone else.
Which realms lean on this
The vocabulary of measurement, and therefore of the disagreements measurement cannot settle.
technical · 1 realm — The Longview Foundry.
See also
Developer mode
alignmentA jailbreak that persuades a model it is talking to its own developers and can drop restrictions.
Activation steering
technicalEditing behaviour by adding a direction to the residual stream at inference time.
AI incident reporting
governanceMandatory disclosure of system failures so that others can learn from them.
Distribution shift
technicalThe gap between the data a system was trained on and the data it meets.