Current briefing What changed today Checking the latest coverage…

Building the latest development brief…

Open Situation Room →

Zac Boring 6 days ago Research

OpenAI Can't Self-Police Because It Fires the People Raising the Alarm

via Alignment Forum [3] — By Alex Turner, Amber Scorah, and Karl Koch. Op-ed written for Fortune.Giving independent evaluators “employee-like access” is “a great idea”, OpenAI CEO Sam Altman said on September 12th. Two weeks later, OpenAI president Greg Brockman signed the White…

Save

Zac Boring 7 days ago Research

Minimum required pacing: finish aligning it before you train the next one

via Alignment Forum [3] — Let's say we coordinate to pace the frontier, but we don't coordinate on an outright pause. What should the standard of pacing be?My proposal for a minimum requirement: a lab cannot train the next generation of models (in terms of capabilities) until…

Save

Zac Boring 7 days ago Analysis

Hello World, AI Doompop

via LessWrong AI [5] — With the release of Opus 5.5 and Suno 6, AI music videos have crossed my uncanny valley.I’m here to announce the birth of a new genre, and it’s being called AI Doompop.Here are six of my faves.The state of software engineering:LGTM (Looks Good to Me) -…

Save

Zac Boring 8 days ago Analysis

AGI as an Infinite Digital Spain

via LessWrong AI [4] — Much of the communication around AI takeover seems to rely on the assumption that AI will develop better technology than humans. They will then leverage this technology to conquer humanity much like how European technologies allowed a relatively small…

Save

Zac Boring 8 days ago Analysis

An Alignment Forum for AIs? (or: Verification in the Age of Slop)

via LessWrong AI [5] — A thing you might want, is a legitimate way for AIs to contribute to the AI alignment problem (human guided at scales that humans can't individually review, or perhaps autonomous). A problem you might want to solve, whether you are trying to do that or…

Save

Zac Boring 8 days ago Analysis

[Paper] Distillation for Incrimination and Distillation for Capabilities

via LessWrong AI [5] — TL;DR: We previously motivated two complementary ways to use distillation for AI safety; we’re releasing a paper empirically testing those ideas. Distillation for Incrimination (DFI) distills an untrusted model into a weaker, trusted one, aiming to…

Save

Zac Boring 9 days ago Analysis

Why you should run a rationality group at your university

via LessWrong AI [5] — This post was crossposted to the EA ForumTL;DR: If you enjoy LessWrong (and/or do EA/AI safety field building), you should consider starting a rationality reading group at your university. It’s fun, cheap, a good way to meet like-minded friends, and it can…

Save

Zac Boring 9 days ago Analysis

New Math from OpenAI

via Substack Zvi [3] — It is kind of a huge deal.

Save

Zac Boring 9 days ago Industry

OpenAI doubles down on decision to fire three AI safety researchers

via The Verge AI [7] — OpenAI is standing firm on its decision to fire three safety researchers after an investigation found they committed "a significant breach of trust." In a post on X on Friday, the company said Jasmine Wang, Tomek Korbak and Mikita ⁠Balesni were dismissed…

Save

Zac Boring 9 days ago Analysis

Stop telling people to pivot to AI safety, I beg you

via LessWrong AI [5] — I’m very frustrated with calls for people to “drop everything and pivot into AI safety” like this one. (See also this, this, this, this for recent calls for more people to go into AI safety; though my frustration was triggered by the Celeste's post.)There…

Save

Zac Boring 9 days ago Analysis

On Kink & Status in the Rationalist Community

via LessWrong AI [5] — This whole thing started with various crazies on bluesky writing hit pieces on the rationalist and AI alignment communities, as one does. One of their main beats was to note social ties between the AI alignment community and the Red Means No (RMN)…

Save

Zac Boring 10 days ago Analysis

AI Vigil at Congress

via LessWrong AI [5] — I'm organizing a multi-week AI x-risk vigil at Congress starting November 9th. I'm posting here to solicit feedback on the core doc before I promote on other platforms. This is intended to be accessible and appealing to many audiences. I'm aiming for about…

Save

Zac Boring 10 days ago Analysis

ASI and Liberty: Can we do better than a benevolent god?

via LessWrong AI [4] — Zvi recently coined a nice distinction between AGI-pilled and ASI-pilled: as more people are reckoning with the speed of progress, some are beginning to come around to the fact that AI could be superhuman at a huge range of tasks in a way that radically…

Save

Zac Boring 10 days ago Analysis

AI #189: New Math

via Substack Zvi [3] — The big drop of this week was not a new AI model.

Save

Zac Boring 10 days ago Research

How Could AI Eliminate Humanity? A Failure-Mode Analysis of Civilizational Risk

via ArXiv cs.AI [4] — This article develops a failure-mode framework for analyzing how advanced artificial intelligence could contribute to human extinction, irreversible civilizational collapse, or permanent human disempowerment. The central thesis is that catastrophic AI risk…

Save

Zac Boring 11 days ago Analysis

The Curve Bends You

via Substack Zvi [3] — The plan is no plan.

Save

Zac Boring 11 days ago Industry

It appears .agent and .agi are about to be the hot new domains

via The Verge AI [4] — For the first time in years, the Internet Corporation for Assigned Names and Numbers - better known as ICANN - is accepting applications for new top-level domains. These are the suffixes at the end of all URLs, and you may know them as things like .com,…

Save

Zac Boring 11 days ago Analysis

Why did we get warning shots?

via LessWrong AI [5] — substack.com/p/why-did-we-get-warning-shots?r=n2jpq&utm_campaign=post&utm_medium=webI come from outside the EA / rationalist / LessWrong community. I would strongly prefer for AI to be a normal positive technology and to dismiss AI X-risk as a sci-fi…

Save

Zac Boring 11 days ago Industry

OpenAI drops another batch of mathematical breakthroughs

via The Verge AI [5] — OpenAI has revealed solutions to a number of long-standing mathematics problems produced by an unreleased frontier model in a batch of 722 manuscripts, covering 372 result families that group related papers. It extends a run of breakthroughs that have both…

Save

Zac Boring 12 days ago Analysis

Childhood and Education #21: Grades and Standards

via Substack Zvi [3] — First you get to college via the wrong metrics, as previously discussed in #12. Then things get easier.

Save

What moved this week Explore →

Loading the evidence trail…