[Paper] Distillation for Incrimination and Distillation for Capabilities
Key takeaway
TL;DR: We previously motivated two complementary ways to use distillation for AI safety; we’re releasing a paper empirically testing those ideas.
Why it's on PDOOM
PDOOM selected this story for its capabilities signals: Capabilities.
CapabilitiesFrom LessWrong AI
TL;DR: We previously motivated two complementary ways to use distillation for AI safety; we’re releasing a paper empirically testing those ideas. Distillation for Incrimination (DFI) distills an untrusted model into a weaker, trusted one, aiming to transfer the teacher's misalignment but not its ability to hide it, so that auditing the student can incriminate the teacher. We test DFI on AuditBench's secret-keeping model organisms, each fine-tuned to have a hidden quirk and trained to deny it. Di
By sebastian_prasanna