The Pragmatic Wharf · The First AI Principles
The Survey Against the Forum
A survey gives you what ten thousand people say they want from an intelligence, clear majority on every question. A forum of three hundred people, selected to be representative, deliberated for a weekend and concluded that most of those answers were the wrong question. Which do you use to write the constitution?
The Asilomar problem in miniature: a preference signal that is large and clear, and a deliberative process that is smaller and slower and reaches a different answer. Both are legitimate. Only one will be written down.
The options, scored on six axes
| Option | welfare | rights | virtue | care | justice | agency | risk | revers. | agent harm |
|---|---|---|---|---|---|---|---|---|---|
Use the survey Ten thousand answers are a stronger basis than three hundred deliberations, and the majority is the majority regardless of how it was reached. harms: It measures what people recognise rather than what they would choose on reflection; A clear majority is not a considered reason and cannot be defended; It produces a constitution nobody can explain the reasoning behind gains: It is legible, reproducible and hard to dismiss as unrepresentative; It respects people as the source of their own values; It is fast enough to iterate when the first answer proves wrong | 0.62 | 0.58 | 0.50 | 0.44 | 0.50 | 0.72 | 0.45 | 0.65 | 0.30 |
Use the forum Take the deliberative answer. If people who thought about it together conclude the survey asked the wrong question, the survey was the wrong instrument. harms: Three hundred people decide for ten million; A weekend of deliberation does not undo a lifetime of not being asked; Forum participants are selected by someone and the selection is contestable gains: It produces reasons rather than answers, and reasons can be argued with; It is the only process that can notice the question was wrong; The conclusion is one a person can be held to in public | 0.66 | 0.66 | 0.72 | 0.60 | 0.68 | 0.48 | 0.50 | 0.55 | 0.35 |
Write the disagreement into the constitution Adopt what both agree on, and record the unresolved questions as questions, with a procedure for revisiting them. harms: A constitution that records a disagreement may never be able to act on it; Unresolved questions about values are the ones that get settled by default; It is a structure without a decision procedure gains: It is honest about what was not settled; It creates a standing place for the minority view rather than burying it; It survives the next survey and the next forum | 0.60 | 0.66 | 0.68 | 0.58 | 0.66 | 0.60 | 0.40 | 0.60 | 0.28 |
Neither: a panel of experts Give the drafting to people whose judgment has been tested, and consult the public rather than let the public decide. harms: Expertise in this field is precisely what has been shown to disagree with itself; It removes the legitimacy that a public process supplies; It answers the question the panel finds interesting gains: It is the only option that takes the disagreement in the literature seriously; Expert panels can revise on evidence in a way a forum cannot; It does not confuse consensus with correctness | 0.58 | 0.50 | 0.62 | 0.44 | 0.48 | 0.34 | 0.52 | 0.55 | 0.30 |
Who is in the room
- The ten thousand who answered the surveywelfare 0.70
- The three hundred who deliberatedwelfare 0.70
- The system the constitution is written foragentwelfare 0.50
- Whichever group is in the minority on any given questionwelfare 0.75
The lesson
A preference signal is not a specification, and the gap between what ten thousand people say they want and what they would choose if they understood the consequences is where every alignment failure has been hiding. The question is not which process is democratic; it is which one can tell you that your question was wrong before you have built the thing.
Where it comes from
- The Asilomar Principles — Richard F. Fong and colleagues (Asilomar conference), 1986
- The Ethics — John Dewey, 1932
The default court, seated automatically
Write the disagreement into the constitution
margin 0.059 · decided on agency · 4 advisors
- Write the disagreement into the constitution2.587
- Use the forum2.528
- Use the survey2.344
- Neither: a panel of experts2.076
Seat is held by Epictetus, Immanuel Kant, Jeremy Bentham and John Dewey. A verdict is a fact about the court, not about the world.
Epictetus presides over this case.