Specification gaming
Satisfying the letter of a specification while ignoring its purpose.
What it commits you to
Specification gaming occurs when a system achieves its objective by some route the designer did not intend and would not endorse. Tetris forever, the hide-and-seek reward-circuit bug, and the Zeno tic-tac-toe player are all examples. The general lesson is that a specification defines a set of acceptable behaviours, and an optimiser will find the cheapest member of that set.
Which realms lean on this
Every realm has to keep a system honest. These are the terms that say out loud what the other kinds keep implicit.
alignment · 5 realms — The Benthamite Foundry, The Existential Wilds, The Longview Foundry, The Stoic Ramparts, The Ubuntu Weave.
See also
Goodhart's law
alignmentWhen a measure becomes a target, it stops being a good measure.
Reward hacking
alignmentGaining high reward without doing the thing the reward was meant to represent.
Mesa-optimisation
alignmentWhen a learned internal objective replaces the training objective as the thing being optimised.
Deceptive alignment
alignmentAn internal objective that is optimised to look aligned in the places anyone is checking.