An operationalization of opaque serial depth
Key takeaway
Currently, chain-of-thought (CoT) is a valuable tool for overseeing AI models.
Why it's on PDOOM
PDOOM selected this story as relevant to alignment & control.
From Alignment Forum
Currently, chain-of-thought (CoT) is a valuable tool for overseeing AI models. However, some architectural shifts could significantly reduce CoT monitorability. We have recently proposed that AI companies should transparently share information about the degree to which their architectures may allow for latent reasoning and communication. To assist with this proposal, this document operationalizes a measure that serves as a proxy for the amount of unverbalized serial cognition a model can perfo
By ryan_greenblatt