AI Governance Scorecard — PDDS pipeline

Open gaps — stated before being asked

• Publishing is fail-open, not fail-closed. A red dbt build still materializes dim_lever, and the export step can write dashboard.json from a failed build. The gates catch violations, but nothing yet prevents a bad note from reaching the page — that enforcement is currently manual discipline. This is the single most important thing to harden.
• No continuous integration. Gates run on a hand-typed dbt build. Until they run automatically on every change and block the deploy, "enforced" depends on the operator remembering to look.
• The judge's validation is real but statistically weak. Cohen's κ = 1.0, but on 12 gold items whose labels co-evolved with the judge, scored by a single model, with no held-out set and no variance runs. eval/VALIDATION.md states this limit plainly; the number is suggestive, not proof.
• The operator agent is sampled, not characterized. Its guardrail (veto of any wall-weakening fix) is deterministic and reliable, but whether the model avoids the cheap fix in the first place has been observed once, not measured across adversarial runs.
• One caveat gate is a weak proxy. assert_required_caveats_present.sql matches a single substring where the concept checker accepts several phrasings, so the keyword gate can red-flag a note that genuinely carries the caveat (and vice versa). Known and documented, not yet reconciled.