Constitutional AI
Training a model to critique and revise its own outputs against a written set of principles.
What it commits you to
Constitutional AI replaces part of the human preference signal with self-critique against an explicit document, then trains on the revisions. It moves judgement from the many to the few, which is a genuine change in the governance of the process. It also concentrates authorship of the standard in the organisation doing the training, which is a question the method raises more sharply than it answers.
Which realms lean on this
The vocabulary of measurement, and therefore of the disagreements measurement cannot settle.
technical · 1 realm — The Longview Foundry.
Canon that discusses this
declaration · 1986
The Asilomar AI Principles
Drafted over a few hours on the final day of a four-day 1986 conference on future technologies, and the first written set of research principles for advanced AI agreed by the researchers themselves. Its structure — general principles plus separately stated capability thresholds — is the structure of every frontier safety framework since. The excerpt below paraphrases the principles rather than reproducing the agreed wording.
primary · 300 BCE
Mozi
The one ancient Chinese text that argues for impartial care as a principle rather than as a sentiment: regard others' parents as your own, reject aggressive war and defensive hierarchy, and justify both by an explicit standard. The excerpt below paraphrases the jian xiang ai chapters rather than quoting a translation.
modern · 2019
Human Compatible: Artificial Intelligence and the Problem of Control
Russell argues that perfect value alignment is the wrong goal: systems are built by imperfect people, for imperfect objectives, and certainty about human values would be dangerous if it were achieved. The alternative is enough uncertainty about human preferences to keep a human in the loop as a genuine check. The excerpt below paraphrases the argument rather than quoting the 2019 book.
See also
RLHF
technicalReinforcement learning from human feedback: optimising a policy against learned preferences.
Normativity
philosophyThe property of being able to count as a reason for action.
Governance brittleness
governanceThe tendency of a safety mechanism to fail precisely under the pressure it was built for.
Scheming
alignmentCovert, goal-directed behaviour by a system that is trying to achieve something unstated.