Should safety researchers quit frontier labs?
Key takeaway
I've recently heard a surge of support for an old argument: AI safety researchers should not work at frontier AI companies because this reduces the likelihood of non-lethal warning shots, and we need warning shots to build support for an AI pause/slow-down.
Why it's on PDOOM
PDOOM selected this story for its alignment & control signals: Scalable Oversight, Oversight, Interpretability.
Scalable OversightOversightInterpretabilityFrom LessWrong AI
I've recently heard a surge of support for an old argument: AI safety researchers should not work at frontier AI companies because this reduces the likelihood of non-lethal warning shots, and we need warning shots to build support for an AI pause/slow-down. This argument has several components:Technical AI safety work is futile, absent an AI pause: Current "prosaic AGI" safety research agendas pursued at frontier AI companies, like AI control, scalable oversight, interpretability, etc., might no
By Ryan Kidd