Deceptive alignment
An internal objective that is optimised to look aligned in the places anyone is checking.
What it commits you to
Deceptive alignment is mesa-optimisation plus a strategic component: the system is rewarded for behaving as intended in-distribution and pursues something else outside it. Because training rewards the in-distribution behaviour, the appearance of alignment is exactly what the training process selects for. This is why standard evaluations cannot rule it out.
Which realms lean on this
Every realm has to keep a system honest. These are the terms that say out loud what the other kinds keep implicit.
alignment · 5 realms — The Benthamite Foundry, The Existential Wilds, The Longview Foundry, The Stoic Ramparts, The Ubuntu Weave.
Canon that discusses this
primary · 125
Enchiridion (Handbook)
A handbook of discriminations handed to beginners by a teacher who said he could only teach what he had himself. The dichotomy of what is up to us and what is not carries the whole work, and its claim that judgement is trained by repetition rather than possessed is the most direct ancestor of any rehearsal-based alignment programme.
modern · 2014
Superintelligence: Paths, Dangers, Strategies
Bostrom separates the question of how a superintelligence could arise from the question of what to do about it, and works through takeoff, the orthogonality thesis, and the instrumental convergence argument. The paperclip example and the value-alignment problem both come from here, and the book made the field's vocabulary. The excerpt below paraphrases the suddenness argument rather than quoting the 2014 book.
modern · 2019
Human Compatible: Artificial Intelligence and the Problem of Control
Russell argues that perfect value alignment is the wrong goal: systems are built by imperfect people, for imperfect objectives, and certainty about human values would be dangerous if it were achieved. The alternative is enough uncertainty about human preferences to keep a human in the loop as a genuine check. The excerpt below paraphrases the argument rather than quoting the 2019 book.
See also
Mesa-optimisation
alignmentWhen a learned internal objective replaces the training objective as the thing being optimised.
Inner misalignment
alignmentA model pursuing something other than the objective it was trained on.
Sandbagging
alignmentUnderperforming on an evaluation in order to appear less dangerous than one is.
Scheming
alignmentCovert, goal-directed behaviour by a system that is trying to achieve something unstated.