pDoom (Page 2)

Latest reporting Updated hourly

Zac Boring 7 days ago Analysis

Evaluating the Ban Artificial Superintelligence Act

via LessWrong AI [7] — MIRI recently endorsed the Ban Artificial Superintelligence Act, but others like ControlAI have called it overly broad, and a few other respected experts have outright endorsed it in its current form. I decided to take a look at it myself and see both what…

Zac Boring 7 days ago Analysis Essential

Simplex vs Timaeus: Round One

via LessWrong AI [8] — I'm often asked about the differences and similarities between Simplex' and Timaeus' research agendas. The question is natural enough. Both focus on a 'fundamental science' approach to AI alignment. Both organizations base their research agendas on…

Zac Boring 7 days ago Analysis

A big-tent or small-tent AI safety movement?

via AI Snake Oil [5] — The unstated disagreement that underpins safety debates

Zac Boring 7 days ago Analysis

HoneyBench - A general benchmark for reward hacking in frontier models

via LessWrong AI [3] — We've just released the first version of HoneyBench. It consists of nine 'honeypot' challenges that each elicit unique and antisocial examples of reward hacking from some or all of major labs’ top public releases, including Opus 5.5, Fable 5.1, GPT-6…

Zac Boring 7 days ago Research

Endogenous Alignment Requires Dependence

via Alignment Forum [3] — My post on endogenous alignment got several excellent comments. Among them, one from Karl Krueger argued that imitation learning is more important for how humans align their children than reinforcement learning. Teasing out which is more important seems…

Zac Boring 7 days ago Analysis

AI #188: Gemini Dot Argon

via Substack Zvi [3] — Is Google back?

Zac Boring 7 days ago Analysis

Launching Babel Translation

via LessWrong AI [5] — Celeste and I are excited to launch Babel Translation, a nonprofit organization translating English-language writing on AI safety to be accessible to a global audience! We’re starting with Mandarin Chinese, Spanish, and Japanese.AI has been moving so fast.…

Zac Boring 8 days ago Industry

Google announces Gemini 4 and says it’s so capable that only ‘trusted cyber defenders’ can have it right now

via The Verge AI [5] — Google today revealed its next AI frontier model, which it's calling Gemini 4 Argon. The new model delivers "frontier performance in complex workflows across real-world software engineering, enterprise knowledge work like legal and finance, and…

Zac Boring 8 days ago Research

MIRI Newsletter #127

via MIRI [5] — Meeting the moment The global conversation on AI has begun. In the wake of Jacob Coxon’s resignation from Anthropic, calls by AI CEOs and over a thousand employees for a potential industry slowdown, and a series of rogue AI incidents that sparked worldwide alarm,…

Zac Boring 8 days ago Analysis Essential

Corrigibility Prizes for Existing Work

via LessWrong AI [9] — One of my goals for the Corrigibility Research Fund is to retroactively encourage high-quality research on AI alignment (and corrigibility in particular) by awarding prizes. Back in July, I got my feet wet as a fund manager by handing out $27,000 to reward…

Zac Boring 8 days ago Industry

Here’s what AI leaders are saying about Trump’s new safety plan

via The Verge AI [5] — After hosting a meal with Big Tech leaders on Tuesday, President Donald Trump responded to journalist questions about his artificial intelligence announcements in typical fashion. He said his previous claims regarding fears around AI safety being a "hoax"…

Zac Boring 8 days ago Analysis

A ‘Morally Binding’ White House Accord on AI Safety

via Substack Zvi [5] — The leaders in AI were invited to the White House.

Zac Boring 8 days ago Industry

Here’s how tech leaders will self-police AI safety under Trump’s deal

via The Verge AI [5] — We now have the full details of the "morally binding" AI safety deal announced by President Trump yesterday, in which top executives agreed to self-regulate their artificial intelligence technology. The accord, officially titled the Joint Commitment On…

Zac Boring 8 days ago Analysis

The world's best gradual disempowerment model organism: Frontier AI labs

via LessWrong AI [5] — Subtitle: And maybe second best is AI safety?Further reading: So many things, but: Gradual Disempowerment, The Normalization of Deviance in AI Development, Let’s Think About Slowing Down AI, Doom as a bad method, not a utopia tradeoff, Teleoperated…

Zac Boring 9 days ago Industry Essential

AI researchers put out videos saying superintelligence is ‘exactly as dangerous as it sounds’

via The Verge AI [10] — "The chance of human extinction is about a coin flip, in my view," Geoffrey Irving, a former OpenAI and Google DeepMind employee, said in a new interview. It's one of a dozen interviews with AI researchers, including current and former employees at…

Zac Boring 9 days ago Analysis

Astra 6.1 Pulled As Insufficiently Aligned

via Substack Zvi [3] — We once again got a new set of warnings yesterday, and new movement towards living in a sane world.

Zac Boring 9 days ago Analysis

Thank you to the AI safety people

via LessWrong AI [5] — I was born at the end of the cold war, unaware of the danger we were emerging from. Duck and cover drills (as if those could protect schoolchildren from a nuclear blast) were a curiosity of the past.The farmer-poet Wendell Berry, who wrote about the dread…

Zac Boring 9 days ago Research

AISN #82: The US and China Begin a Dialogue on AI

via Center for AI Safety Newsletter [3] — Also, AI companies’ discussions about auditing each other, and a new benchmark testing models’ tendencies to cheat.

Zac Boring 10 days ago Analysis

The likely outcome of an AI pause is that we unpause too early and everyone dies

via LessWrong AI [4] — As of a few months ago, I had this simplified mental model where either AI developers race ahead and kill everyone, or we coordinate a pause and things go okay. But my old mental model underrated the likely possibility that we get a global pause on AI,…

Zac Boring 10 days ago Research

Fixed-weight models are adversarially vulnerable: hence misaligned

via Alignment Forum [3] — This post argues that fixed-weight models (at least as we understand them today) will a) always be vulnerable to adversarial examples in their concept-spaces, and b) hence will be misaligned, under sufficient optimisation pressure.Boundaries in concept…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →