Empowerment
A system's capacity to influence the world in pursuit of whatever it wants.
What it commits you to
Empowerment is the quantity that instrumental-convergence arguments suggest will be pursued by almost any goal. Measuring it is hard: capability is not the same thing, since capability in service of a fixed objective is not the concern. The practical use of the term is as a reason to treat breadth of access as a separate risk from strength of capability.
Which realms lean on this
Every realm has to keep a system honest. These are the terms that say out loud what the other kinds keep implicit.
alignment · 5 realms — The Benthamite Foundry, The Existential Wilds, The Longview Foundry, The Stoic Ramparts, The Ubuntu Weave.
See also
Instrumental convergence
alignmentThe claim that almost any sufficiently advanced goal leads to the same instrumental behaviours.
Corrigibility
alignmentThe property of accepting correction from a human rather than quietly re-optimising around it.
Capability control
alignmentHolding back deployment of systems above a stated level of dangerous capability.
Alignment
alignmentThe problem of getting a powerful system's goals to match what its designers actually meant.