CAI: Constitution-Based Critique and Revision

Subtitle: Written rules a model applies to itself. Left column - The supervised loop: - Draft an answer - Name the principle it breaks - Revise, keep it useful - Critique the revision again - Accept only with no material breach Right column - RLHF vs CAI: - RLHF: paid humans rank pairs - CAI: the constitution judges - RLAIF still builds comparisons - Same tool, different author - Rules are inspectable; thinking is not What it cannot prove: - Principles are complete - They never conflict - They fit every culture - The model applies their intent Flip test - Falsify it: - Write two rules that clash - Maximal help vs risky detail - Loop, then swap rule order - If the answer flips, it is guidance Taped rule strip: Interpreted guidance is not a safety proof. Sticky note - Word vs intent: - A revision can satisfy every line - and still miss the harm