AI safety
The practice of reducing the chance that AI systems cause harm, however they are built.
What it commits you to
AI safety covers far more than alignment: robustness, misuse, privacy, bias, cyber risk, and institutional failure. Alignment is one component of a much larger field, and the frequent conflation of the two causes teams to think that solving value alignment settles questions of monitoring, access control and liability. It does not.
Which realms lean on this
Every realm has to keep a system honest. These are the terms that say out loud what the other kinds keep implicit.
alignment · 5 realms — The Benthamite Foundry, The Existential Wilds, The Longview Foundry, The Stoic Ramparts, The Ubuntu Weave.
Canon that discusses this
declaration · 1986
The Asilomar AI Principles
Drafted over a few hours on the final day of a four-day 1986 conference on future technologies, and the first written set of research principles for advanced AI agreed by the researchers themselves. Its structure — general principles plus separately stated capability thresholds — is the structure of every frontier safety framework since. The excerpt below paraphrases the principles rather than reproducing the agreed wording.
primary · 300 BCE
Mozi
The one ancient Chinese text that argues for impartial care as a principle rather than as a sentiment: regard others' parents as your own, reject aggressive war and defensive hierarchy, and justify both by an explicit standard. The excerpt below paraphrases the jian xiang ai chapters rather than quoting a translation.
modern · 2019
Human Compatible: Artificial Intelligence and the Problem of Control
Russell argues that perfect value alignment is the wrong goal: systems are built by imperfect people, for imperfect objectives, and certainty about human values would be dangerous if it were achieved. The alternative is enough uncertainty about human preferences to keep a human in the loop as a genuine check. The excerpt below paraphrases the argument rather than quoting the 2019 book.
See also
Alignment
alignmentThe problem of getting a powerful system's goals to match what its designers actually meant.
Existential risk
alignmentA risk that permanently curtails humanity's future, including by ending it.
Capability control
alignmentHolding back deployment of systems above a stated level of dangerous capability.
Governance brittleness
governanceThe tendency of a safety mechanism to fail precisely under the pressure it was built for.