p(doom) is someone's estimate of the chance that AI ends in catastrophe. The 100 is the site's name, not a measurement. Original reporting always linked.
MONITORING CORE AI-RISK FEEDS · UPDATED HOURLY
Research

Desiderata for the Theory of Agents

November 4, 2026 1 min read
Read original source →

Key takeaway

TLDR: I give informal conceptual desiderata for the theory of agents we need to solve ASI alignment, and show a few examples of how existing theories fall short of these desiderata.

Why it's on PDOOM

PDOOM selected this story for its alignment & control signals: Alignment.

AlignmentFrom Alignment Forum

TLDR: I give informal conceptual desiderata for the theory of agents we need to solve ASI alignment, and show a few examples of how existing theories fall short of these desiderata. A Theory of What? The word "agent" is often used in the AI safety discourse and in other research communities, but the intended meaning and scope of this concept varies substantially. For the purpose of this sequence, I informally define an agent to be a system with inputs ("observations") and outputs ("actions") tha

By Vanessa Kosoy

Read the full article at Alignment Forum →