DOOM LEVEL
--
%
Latest Headlines
Auto-Updated
ControlAI 2025 Impact Report
via LessWrong AI [4] — This post highlights a few key excerpts from our full impact report. You can read the full report at https://controlai.com/impact-report-2025.ControlAI is a non-profit organization working to avert the extinction risks posed by superintelligence. We help…
Anthropic vs. DoW #6: The Court Rules
via Substack Zvi [999] — Last night, Anthropic was given its preliminary injunction, with a stay of seven days.
Trust as Monitoring: Evolutionary Dynamics of User Trust and AI Developer Behaviour
via ArXiv cs.AI [8] — AI safety is an increasingly urgent concern as the capabilities and adoption of AI systems grow. Existing evolutionary models of AI governance have primarily examined incentives for safe development and effective regulation, typically representing users'…
ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence
via ArXiv cs.AI [4] — We introduce ARC-AGI-3, an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments in which agents must explore, infer goals, build internal models of environment dynamics, and plan effective action sequences…
My hobby: running deranged surveys
via LessWrong AI [4] — In late 2024, I was on a long walk with some friends along the coast of the San Francisco Bay when the question arose of just how much of a bubble we live in. It’s well known that the Bay Area is a bubble, and that normal people don’t spend that much time…
Sen. Sanders (I-VT) and Rep. Ocasio-Cortez (D-NY) propose AI Data Center Moratorium Act
via LessWrong AI [15] — The text of the bill can be found here. It begins by citing the warnings of AI company CEOs and deep learning pioneers Geoffrey Hinton and Yoshua Bengio, the 2023 FLI open letter calling for a 6-month pause, and the 2025 FLI statement on…
Test your best methods on our hard CoT interp tasks
via Alignment Forum [999] — Authors: Daria Ivanova, Riya Tyagi, Arthur Conmy, Neel NandaDaria and Riya are co-first authors. This work was done during Neel Nanda’s MATS 9.0. Claude helped write code and suggest edits for this post.TL;DR One of our best safety techniques right…
AI #161 Part 1: 80,000 Interviews
via Substack Zvi [999] — The major technical advances this week were in agentic coding, as covered yesterday.
A Toy Environment For Exploring Reasoning About Reward
via Alignment Forum [999] — tldr: We share a toy environment that we found useful for understanding how reasoning changed over the course of capabilities-focused RL. Over the course of capabilities-focused RL, the model biases more strongly towards reward hints over direct…
$1 billion is not enough; OpenAI Foundation must start spending tens of billions each year
via LessWrong AI [6] — OpenAI is now a public benefit corporation, with a charter that demands they use AGI for the benefit of all, and do so safely. To justify this structure to the Attorneys General of Delaware and California, they split off the nonprofit OpenAI Foundation,…
Claude Code, Cowork and Codex #6: Claude Code Auto Mode and Full Cowork Computer Use
via Substack Zvi [999] — Whatever else you think about Anthropic’s agentic coding department, they ship.
Agentic commerce runs on truth and context
via MIT Technology Review [4] — Imagine telling a digital agent, “Use my points and book a family trip to Italy. Keep it within budget, pick hotels we’ve liked before, and handle the details.” Instead of returning a list of links, the agent assembles an itinerary and executes…
The AI Hype Index: AI goes to war
via MIT Technology Review [4] — AI is at war. Anthropic and the Pentagon feuded over how to weaponize Anthropic’s AI model Claude; then OpenAI swept the Pentagon off its feet with an “opportunistic and sloppy” deal. Users quit ChatGPT in droves. People marched through London in…
Intelligence Inertia: Physical Principles and Applications
via ArXiv cs.AI [3] — While Landauer's principle establishes the fundamental thermodynamic floor for information erasure and Fisher Information provides a metric for local curvature in parameter space, these classical frameworks function effectively only as approximations within…
Introducing the OpenAI Safety Bug Bounty program
via OpenAI Blog [7] — OpenAI launches a Safety Bug Bounty program to identify AI abuse and safety risks, including agentic vulnerabilities, prompt injection, and data exfiltration.
The Fourth World
via LessWrong AI [4] — Is consciousness the last moral world?Imagine trying to explain to a virus why suffering matters.A virus is a simple self-replicating molecule: unsophisticated and arguably not even alive. It has no experience. It just copies itself according to chemical…
Arm’s first CPU ever will plug into Meta’s AI datacenters later this year
via The Verge AI [4] — After decades of only licensing its chip designs for others to use, UK-based Arm revealed the first chip it's producing on its own, and the first customer. Dubbed the Arm AGI CPU, it's another chip designed for inference, or running the cloud processing…
Book Review: Open Socrates (Part 2)
via Substack Zvi [999] — Yesterday I posted Part 1. Read that first. This is Part 2 of 2.
The AIXI perspective on AI Safety
via LessWrong AI [5] — I am also discussing something that is still a bit speculative, since we do not yet have ASI. While basic knowledge of AIXI is the only strict prerequisite, I suggest reading cognitive tech from AIT before this post for context.AIXI is often used as a…
Measuring and improving coding audit realism with deployment resources
via LessWrong AI [5] — TL;DR We study realism win rate, a metric for measuring how distinguishable Petri audit transcripts are from real deployment interactions. We use it to evaluate the effect of giving the auditor real deployment resources (system prompts, tool definitions,…
Live Doom Meter
--
%
0% — We're fine
100% — GG
The Doom Meter is a composite score derived from prediction markets and feed sentiment, updated daily.
70%
Prediction Markets
Weighted average of Manifold Markets questions on AI catastrophe, AGI timelines, expert surveys, and key figures. Direct doom indicators weighted higher than indirect capability markers.
30%
Feed Sentiment
Percentage of recent headlines containing high-alarm keywords (existential risk, catastrophe, extinction). Higher alarm density = higher score.
This is not a scientific estimate of existential risk. It is an opinionated, transparent signal — a vibes-based thermometer for AI doom discourse.
P(Doom) Scoreboard
0%25%50%75%100%
Loading estimates...
Recent Voices
We are creating something that will be more powerful than us. I don't know a good precedent for a less intelligent thing managing a more intelligent thing.
— Geoffrey Hinton, Nobel Prize Lecture, Dec 2024
If you're not worried about AI safety, you're not paying attention.
— Sen. Blumenthal, Senate AI Hearing, 2024
The probability of doom is high enough that we should be working very hard to reduce it.
— Yoshua Bengio, MILA Talk, 2024