Governance brittleness
The tendency of a safety mechanism to fail precisely under the pressure it was built for.
What it commits you to
A mechanism is brittle when the conditions that trigger it are the same conditions under which it stops being enforced — a crisis, a competitive emergency, an insider with authority. Tripwires, review boards, and internal red teams all share this structure. The countermeasure is external binding, which is also why internal governance documents are usually described as evidence rather than assurance.
Which realms lean on this
Procedure, not conviction. This is the vocabulary of constitutions, audits and the people who have to sign.
governance · 2 realms — The Contractual Court, The Ubuntu Weave.
Canon that discusses this
modern · 2002
No Future Without a Past
Tutu argues from ubuntu — that a person becomes fully human through relationship with other people — to a politics of shared responsibility rather than blame. The formulation that matters for a governance question is the negative one: a future without the past that made it is not a future anyone can build on. The excerpt below paraphrases the essay rather than quoting it.
modern · 1950
The Human Use of Human Beings
Wiener applies the theory of feedback and control to law, politics and medicine, and argues that communication is more fundamental than command. The book that gave cybernetics its name in 1948 becomes, two years later, the founding text of the ethics of steering large systems without crushing what they steer. The excerpt below paraphrases the book rather than quoting it.
See also
Tripwire
alignmentA pre-committed threshold that triggers an automatic slowdown rather than a new debate.
External auditing
governanceIndependent verification of a developer's claims by an accredited third party.
Frontier safety framework
governanceA developer's published promise of what it will do at each capability threshold.
AI safety case
governanceA structured, auditable argument that a system's residual risk is acceptable.