Regulatory capture
A regulator becoming aligned with the interests it is meant to constrain.
What it commits you to
Capture happens when the regulated party supplies the expertise, the funding and the information the regulator needs, and the regulator's incentives reward good relations with the regulated. It is the default failure mode for any governance body whose subject is technically sophisticated and commercially urgent, which is a description of AI safety governance entirely.
Which realms lean on this
Procedure, not conviction. This is the vocabulary of constitutions, audits and the people who have to sign.
governance · 2 realms — The Contractual Court, The Ubuntu Weave.
See also
External auditing
governanceIndependent verification of a developer's claims by an accredited third party.
Governance brittleness
governanceThe tendency of a safety mechanism to fail precisely under the pressure it was built for.
Frontier developer
governanceAn organisation training systems at the edge of available capability.
Institutionalisation
governanceA principle hardened into a rule enforced by an organisation that outlives its authors.