pDoom (Page 46)

Latest reporting Updated hourly

Zac Boring 7 months ago Analysis

Types of Handoff to AIs

via LessWrong AI [4] — This is a rough draft I'm posting here for feedback. If people like it, a version of it might make it into the next scenario report we write....We think it’s important for decisionmakers to track whether and when they are handing off to AI systems. We…

Zac Boring 7 months ago Analysis

You can’t imitation-learn how to continual-learn

via LessWrong AI [5] — In this post, I’m trying to put forward a narrow, pedagogical point, one that comes up mainly when I’m arguing in favor of LLMs having limitations that human learning does not. (E.g. here, here, here.)See the bottom of the post for a list of subtexts that…

Zac Boring 7 months ago Analysis Essential

AICRAFT: DARPA-Funded AI Alignment Researchers — Applications Open

via LessWrong AI [9] — AICRAFT: DARPA-Funded AI Alignment Researchers — Applications OpenTL;DR: We hypothesize that most alignment researchers have more ideas than they have engineering bandwidth to test. AICRAFT is a DARPA-funded project that pairs researchers with a fully…

Zac Boring 7 months ago Analysis Essential

Terrified Comments on Corrigibility in Claude's Constitution

via LessWrong AI [9] — (Previously: Prologue.) Corrigibility as a term of art in AI alignment was coined as a word to refer to a property of an AI being willing to let its preferences be modified by its creator. Corrigibility in this sense was believed to be a desirable but…

Zac Boring 7 months ago Analysis Essential

We Started Lens Academy: Scalable Education on Superintelligence Risk

via LessWrong AI [9] — The number of people who deeply understand superintelligence risk is far too small. There's a growing pipeline of people entering AI Safety, but most of the available onboarding covers the field broadly, touching on many topics without going deep on the…

Zac Boring 7 months ago Analysis Essential

Monthly Roundup #40: March 2026

via Substack Zvi [999] — It is that time again.

Zac Boring 7 months ago Analysis

Bridge Thinking and Wall Thinking

via LessWrong AI [5] — There are a couple of frames I find useful when understanding why different people talk very differently about AI safety - the wall, and the bridge.A wall is incrementally useful. Every additional brick you add is good, and the more bricks you add the…

Zac Boring 7 months ago Analysis

Extracting Performant Algorithms Using Mechanistic Interpretability

via LessWrong AI [7] — A Prequel: The Tree of Life Inside a DNA Language ModelLast year, researchers at Goodfire AI took Evo 2, a genomic foundation model, and found, quite literally, the evolutionary tree of life inside. The phylogenetic relationships between thousands of…

Zac Boring 7 months ago Research Essential

Operationalizing FDT

via Alignment Forum [999] — This post is an attempt to better operationalize FDT (functional decision theory). It answers the following questions:given a logical causal graph, how do we define the logical do-operator?what is logical causality and how might it be formalized?how…

Zac Boring 7 months ago Analysis

Ideologies Embed Taboos Against Common Knowledge Formation: a Case Study with LLMs

via LessWrong AI [4] — LLMs are searchable holograms of the text corpus they were trained on. RLHF LLM chat agents have the search tuned to be person-like. While one shouldn't excessively anthropomorphize them, they're helpful for simple experimentation into the latent…

Zac Boring 7 months ago Analysis Essential

Why AI Evaluation Regimes are bad

via LessWrong AI [9] — How the flagship project of the AI Safety Community ended up helping AI Corporations.I care about preventing extinction risks from superintelligence. This de facto makes me part of the “AI Safety” community, a social cluster of people who care about these…

Zac Boring 7 months ago Analysis Essential

AI #159: See You In Court

via Substack Zvi [999] — The conflict between Anthropic and the Department of War has now moved to the courts, where Anthropic has challenged the official supply chain risk designation as well as the order to remove it from systems across the government, claiming retaliation for…

Zac Boring 7 months ago Analysis

Dwarkesh Patel on the Anthropic DoW dispute

via LessWrong AI [3] — Below is the text of blog post that Dwarkesh Patel wrote on the Anthropic DoW dispute and related topics. He has also narrated it here. By now, I’m sure you’ve heard that the Department of War has declared Anthropic a supply chain risk, because Anthropic…

Zac Boring 7 months ago Research Essential

How well do models follow their constitutions?

via Alignment Forum [999] — This work was conducted during the MATS 9.0 program under Neel Nanda and Senthooran Rajamanoharan.There's been a lot of buzz around Claude's 30K word constitution ("soul doc"), and unusual ways Anthropic is integrating it into training.If we can…

Zac Boring 7 months ago Analysis

How Hard a Problem is Alignment? (My Opinionated Answer)

via LessWrong AI [6] — TL;DR: Comparing person-years of effort, I argue that AI Safety seems harder than for steam engines, but probably less hard than the Apollo program or . I discuss why I suspect superalignment might not be super-hard. My has come down over the last…

Zac Boring 7 months ago Industry

Grammarly says it will stop using AI to clone experts without permission

via The Verge AI [4] — Superhuman says it has disabled Grammarly's "expert review" AI feature that said its edit suggestions were "inspired by" real writers, including our editor-in-chief and other Verge staff members. "After careful consideration, we have decided to disable…

Zac Boring 7 months ago Industry

Canva’s new editing tool adds layers to AI-generated designs

via The Verge AI [4] — Canva introduced a new feature that separates flat image files and AI-generated visuals into layered, fully editable designs. The Magic Layers tool is launching in public beta today in the US, UK, Canada, and Australia, allowing design components like…

Zac Boring 7 months ago Analysis Essential

GPT-5.4 Is A Substantial Upgrade

via Substack Zvi [999] — Benchmarks have never been less useful for telling us which models are best.

Zac Boring 7 months ago Research Essential

The Refined Counterfactual Prisoner's Dilemma

via Alignment Forum [999] — I was inspired to revise my formulation of this thought experiment by Ihor Kendiukhov's post On The Independence Axiom.Kendiukhov quotes Scott Garrabrant:My take is that the concept of expected utility maximization is a mistake. [...] As far as I…

Zac Boring 7 months ago Analysis

The Day After Move 37

via LessWrong AI [4] — I was a few months into 21 years old when a hijacked plane crashed into the first World Trade Center tower. I was commuting in to work listening to the radio (as was the style at the times). I couldn’t figure out how the heck a plane could hit the tower.…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →