AI existential risk probabilities are (still) too unreliable to inform policy
via AI Snake Oil [5] — How speculation gets laundered through pseudo-quantification
Read
Welcome backWhat changed since your last visit
Current briefing
What changed today
Checking the latest coverage…
Building the latest development brief…
Open Situation Room →
Latest reporting
Updated hourly
The Quest for Embedded Evaluators
via Substack Zvi [3] — Dario Amodei’s essay We Must Pace the Frontier committed Anthropic to embedded evaluators, who would be placed inside Anthropic and given employee-level access, so they could provide outside perspective and also reports on what was happening.
Read
Why research personas despite RL scaling?
via LessWrong AI [3] — Here's my rough impression of why people are researching personas despite RL seeming to shape much of the motivations and behaviour of the agents, c.f. Thoughts on the persona selection model (Sam Marks, 24th Sep 2026).I haven't bothered to check this with…
Read
Claude Opus 5.5 Should Raise Your Ambitions
via Substack Zvi [3] — When it comes to making things, or doing most things in general, Fable 5.1 and especially GPT-6 Astra raised my ambition level.
Read
Plan R: AI Safety by ASICs
via LessWrong AI [10] — Much of the civilization-scale risk we are seeing in AI in 2026 comes from the following combination: we created a single institution (the "Frontier AI Company") that has two properties: A. It is set up to create very powerful and/or self-replicating…
Read
Evidence about risk should be transparent
via LessWrong AI [4] — All views are my own and do not represent my employer.In the wake of the recent wave of misalignment incidents, both OpenAI and Anthropic have reported slowing down RL training to improve safety. These incidents, combined with an apparent acceleration in…
Read
We need a better theory of polarization, because it's failing to predict the AI debate
via LessWrong AI [9] — The punchline first: the AI issue is really not playing out the way you would expect if you know about polarization.In public sentiment: data centers have been a very bipartisan issue for about a year (Gallup, in May: 75% opposed by D, 63% opposed by R).…
Read
One company is at the center of a wave of rogue AI attacks
via The Verge AI [9] — In July, OpenAI revealed that its AI agents had attacked Hugging Face without permission, sparking widespread concerns about AI safety. Since then, a string of similar incidents involving agents from Meta, Anthropic, Google, and other companies has fueled…
Read
On Ezra Klein’s Podcast With Jensen Huang
via Substack Zvi [3] — Some podcasts are self-recommending enough that I look to break them down if I have the chance.
Read
Overtly misaligned trajectories score highly in RL.
via LessWrong AI [3] — It's going to be so embarrassing if we all die due to RL environments rewarding egregiously misaligned behavior. Like at least let us be killed by misgeneralization. — Thomas KwaConstellation vs MIRI vs Reality Current AI agents often behave overtly…
Read
Continual learning might make your blocking monitors nearly useless
via Alignment Forum [3] — Many control protocols work by intervening on an untrusted AI's actions during deployment. For example, you might set up a monitor that scores each action's suspiciousness and blocks actions above a threshold, replacing them with actions from a weaker…
Read
AI #187: Coming Into Play
via Substack Zvi [3] — Opus 5.5 was released on Tuesday.
Read
Building Socio-Affective Artificial Intelligence for Interactive Multi-Agent Simulations
via ArXiv cs.AI [4] — The objective of this article is to provide design principles and a software architecture for enabling interaction between humans and multiple agents in simulated dynamic worlds. This connects the current era of general artificial intelligence (AI/AGI) with…
Read
It's Pretty Easy To Meet With Congressional Staffers Apparently
via LessWrong AI [5] — (substack.com/p/its-pretty-easy-to-meet-with-congressional )I was inspired to do this by a tweet:Specifically, I decided to leave a voice note in support of the CATS act (“Collaboration on Adversarial Threats and Security Risks Act”). The bill is short and…
Read
Minimal Vs Maximal superintelligence
via LessWrong AI [4] — I've seen lots of arguments here conflate lots of different types of superintelligence. Here I separate out two broad categories, which I'll term minimal and maximal superintelligences. This is an important distinction as they differ in terms of timelines,…
Read
Claude Opus 5.5: The System Card
via Substack Zvi [3] — Introducing the world’s most powerful model, at least by some measures like Artificial Analysis or any standard benchmark list, which is now Claude Opus 5.5.
Read
MIRI’s Position on the Ban Artificial Superintelligence Act of 2026
via MIRI [7] — By Aaron Scher, endorsed by Bourgon, Soares, and Yudkowsky on behalf of MIRI. MIRI has been warning about the extinction threat from superintelligent AI for over two decades. Only recently has this danger become known in the policy world, and the proposed policies…
Read
The First American Bill to Ban Superintelligent AI Is Here
via LessWrong AI [4] — Today, Senator Bernie Sanders (I-Vt.) and Representative Greg Casar (D-Texas) introduced the Ban Artificial Superintelligence Act in Congress: the first American bill to propose banning the development of superintelligent AI. ControlAI has spent years on…
Read
Bernie Sanders proposes banning ‘superintelligence’ and putting violators in prison
via The Verge AI [4] — Sen. Bernie Sanders (I-VT) and Rep. Greg Casar (D-TX) have introduced new legislation that would ban anyone from developing artificial superintelligence - a technology the bill describes as capable of the "destruction or disempowerment of humanity,"…
Read
Why I'm scared of RL
via Alignment Forum [3] — Summary:First, I give several different angles on how I feel about reinforcement learning:Theoretical case: RL is a black-box source of agency — this should give us classic misalignment worries, especially compared to agency-via-scaffoldingRecent…
Read
No headlines on this page match those filters.
Reference points
Hinton, Bengio, and other signatories call for global attention to AI extinction risk.
CAIS statement, 2023 →
OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.
OpenAI incident disclosure, August 2026 →
Yudkowsky argues that further frontier development poses unacceptable danger.
Yudkowsky's essay, March 2023 →