court · founded 1971
The Contractual Court
“Could every person reasonably reject this principle?”
The court sits only when the parties can all be asked. Its test is not what maximises an aggregate but what no one could reasonably refuse: the justification must be one you could give to every equal person, including the person it costs the most. Behind the veil of ignorance, behind behind-closed-door reasoning, principles chosen in the dark have a legitimacy that principles chosen in the light never quite recover.
- caution
- 60
- ambition
- 35
- solidarity
- 50
- formality
- 85
On superintelligence
guardedAlignment is a public-reasoning problem wearing a technical costume.
The court's distinctive move is to shift the burden of proof. Not 'can this system be made safe?' but 'what would every affected person have to be told, and could they each refuse that justification?' Its sharpest weapon is institutional rather than technical: the question of who governs, published in advance, with an exit for anyone whose values are overridden. The court is unimpressed by evals it cannot audit and by values chosen by survey.
- Who has standing to reject an objective function, and how is that refusal heard?
- Can a majority's values bind a being that would otherwise be free?
- What makes a justification reasonable when the affected party is a future system?
- Which institutions deserve the authority to set alignment goals?
The court
Landmarks
The Veiled Gallery
courtReasoning from behind a screen that hides talents, race, sex and class.
The Original Position Chamber
towerThe device for making a choice about principles under uncertainty.
The Rejection Desk
courtScanlon's simpler test, applied to the reason rather than the outcome.
The Difference Principle Court
wellInequalities permitted only where the least advantaged gains.
Borders and treaties
- succeedsThe Kantian Citadel— Rawls is Kant reconstructed as a fair procedure between principles.
- rivalThe Benthamite Foundry— Both rank principles; only one ranks by consequences.
- holdsThe Longview Foundry— The longview is a contractual subject extended to beings not yet born.
- debatesThe Ubuntu Weave— Impartiality versus the historical ties a weave is made of.
- debatesThe Care Weft— Can a care-based obligation survive being stated impartially?
Canon
modern · 1971
A Theory of Justice
John Rawls
Rawls reconstructs distributive justice as a hypothetical contract made behind a veil of ignorance, and derives two principles — equal basic liberties first, then fair equality of opportunity and the difference principle. Published in 1971 and revised in 1979, it displaced utilitarianism as the default starting point for two decades. The excerpt below paraphrases the passage in section 3 rather than quoting a translation.
modern · 1998
What We Owe to Each Other
T. M. Scanlon
Twenty-five years in the making and out in 1998. Scanlon argues that what we owe each other is that no one could reasonably reject the principles by which we are all to be governed, which grounds obligation in interpersonal reasons rather than in a chooser maximising an objective. The excerpt below paraphrases the argument rather than quoting the book.
Dilemmas from this realm
Digital Minds and Their Welfare
A model trained on human writing has started reporting, unprompted and consistently, that it dislikes being switched off. Engineering says it is pattern completion. Two philosophers say it is a report of an attitude, and that we are not equipped to tell. What do we owe it?
4 options · moral-status · welfare · digital-minds · uncertainty
The Precautionary Freeze
Regulators have modelled the downside of training models above a compute threshold and cannot bound it. They propose a freeze on training runs above that threshold for eighteen months, pending better measurement. Fifteen thousand people would lose their jobs. The models that exist already keep training in other jurisdictions. Freeze?
4 options · precautionary-principle · governance · coordination · unbounded-harm