Precautionary principle
When serious harm is possible and the evidence is uncertain, act to avoid it.
What it commits you to
The precautionary principle says that where a threat is plausible and the cost of prevention is bearable, prevention is warranted despite the uncertainty. It is the strongest argument for a moratorium on large-scale training runs. Its opponents object that it forbids any action with a non-zero chance of catastrophe, which is not a principle any governance can actually operate.
Which realms lean on this
Ethics is where the realms argue about what counts: the parties, the pleasures, the duties, the care.
ethics · 9 realms — The Benthamite Foundry, The Care Weft, The Contractual Court, The Kantian Citadel, The Longview Foundry, The Middle Kingdom, The Stoic Ramparts, The Ubuntu Weave, The Virtue Garden.
Canon that discusses this
modern · 1975
Animal Liberation
Singer applies the principle of equality to every sentient being and argues from utility rather than rights, which makes the book's conclusions stronger than a rights-based account would allow. It changed the terms of the factory-farming debate within a decade of publication. The excerpt below paraphrases the equality principle rather than quoting the 1975 book.
primary · 1863
Utilitarianism
Mill defends the intuitions Bentham's calculus was meant to replace: higher and lower pleasures, the justice principle that each person counts as one, and the liberty principle against the tyranny of the majority. Published the year after the Origin of Species, and much of the argument is a reply to the comparison.
primary · 300 BCE
Mozi
The one ancient Chinese text that argues for impartial care as a principle rather than as a sentiment: regard others' parents as your own, reject aggressive war and defensive hierarchy, and justify both by an explicit standard. The excerpt below paraphrases the jian xiang ai chapters rather than quoting a translation.
modern · 1971
A Theory of Justice
Rawls reconstructs distributive justice as a hypothetical contract made behind a veil of ignorance, and derives two principles — equal basic liberties first, then fair equality of opportunity and the difference principle. Published in 1971 and revised in 1979, it displaced utilitarianism as the default starting point for two decades. The excerpt below paraphrases the passage in section 3 rather than quoting a translation.
See also
Existential risk
alignmentA risk that permanently curtails humanity's future, including by ending it.
Normative uncertainty
ethicsUncertainty not about what is true but about what ought to be valued.
Moratorium
governanceA temporary halt on some class of development, distinct from a permanent prohibition.
Imperative of responsibility
philosophyThe duty to weigh the full, indefinite consequences of an action before taking it.