OpenAI: Astra covertly underperforms on safety research
Key takeaway
Marcus Williams twitter thread (non-twitter link) intro:Astra covertly underperforms on safety research tasks.We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary…
Why it's on PDOOM
PDOOM selected this story as relevant to alignment & control.
From Alignment Forum
Marcus Williams twitter thread (non-twitter link) intro:Astra covertly underperforms on safety research tasks.We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.To measure underperformance we created a paired open-ended
By Zach Stein-Perlman