Continual learning might make your blocking monitors nearly useless
Key takeaway
Many control protocols work by intervening on an untrusted AI's actions during deployment.
Why it's on PDOOM
PDOOM selected this story as relevant to alignment & control.
From Alignment Forum
Many control protocols work by intervening on an untrusted AI's actions during deployment. For example, you might set up a monitor that scores each action's suspiciousness and blocks actions above a threshold, replacing them with actions from a weaker "trusted" model (a defer-to-trusted protocol). Interventions like this cost usefulness: the protocol sometimes replaces good actions with worse ones.Continual learning, by which I mean any mechanism that lets an AI accumulate skills and knowledge o
By Alex Mallen