External auditing
Independent verification of a developer's claims by an accredited third party.
What it commits you to
External auditing gives evaluation access to parties with an incentive to disagree, which is the missing element in internal review. Its difficulties are known: the auditor must be accredited, the results must be publishable, and the subject must not be able to shop for a friendly assessor. All three are institutional problems rather than technical ones.
Which realms lean on this
Procedure, not conviction. This is the vocabulary of constitutions, audits and the people who have to sign.
governance · 2 realms — The Contractual Court, The Ubuntu Weave.
See also
AI safety case
governanceA structured, auditable argument that a system's residual risk is acceptable.
Capability threshold
governanceA pre-specified level of capability that triggers an obligation or a prohibition.
Regulatory capture
governanceA regulator becoming aligned with the interests it is meant to constrain.
AI incident reporting
governanceMandatory disclosure of system failures so that others can learn from them.