DOOM LEVEL
--
%
Latest Headlines
Auto-Updated
Superhuman Articulacy as an LLM Safety Target
via LessWrong AI [6] — TL;DR: Current LLMs are bad communicators relative to their agentic capabilities. I claim that articulacy is useful (and perhaps necessary) for AI safety and suggest a path for improving articulacy.Briefly: a theory for articulacyFrequently, LLM agents…
No Space Like J-Space
via Substack Zvi [999] — There is a new very cool Anthropic paper: Verbalizable Representations Form a Global Workspace in Language Models. You can read the blog post verison here.
Meta’s new Muse Image model can pull other Instagram users into AI photos
via The Verge AI [4] — Meta is launching the first AI image generation model made by its Superintelligence Labs division. The Muse Image model now powers the image-making tools across the Meta AI app, Instagram, and WhatsApp, and it's coming soon to Facebook and Messenger,…
Current views on large-scale longtermist philanthropy
via LessWrong AI [5] — This post summarizes some of my views. It states uncertain things boldly in part to [surface disagreements / help readers point out things that seem wrong or missing]. Note that many things are uncertain and many claims are not justified here.One tl;dr:…
Data filtering works a lot worse than you would expect
via Alignment Forum [999] — This work was largely done during Neel Nanda's MATS 10.0 Exploration Phase. J Rosser and Dohun Lee are co-first authors for this post with equal contribution. Josh Engels and Neel Nanda supervised the project, and provided guidance and feedback…
Visioning: Concretely Imagining What You Want
via LessWrong AI [4] — When John told me (Gretta) his practice of “visioning,” I was skeptical at first. I gave it a try, a little bit out of spite, to show him I was capable of it.It blew me away.Here’s what I told him about it a few days later, edited for brevity and…
SFF is very suboptimal
via LessWrong AI [5] — I recently served as a recommender in SFF's annual funding round (grants will be decided and announced in September). I'm deeply grateful for SFF's funders, and I hope more AI safety donors appear in the future.Unfortunately, the SFF experience is bad for…
Tie training can make DPO/RLHF-trained AIs generalize better
via LessWrong AI [3] — This post covers our recent ICML paper: Spurious Correlation Learning in Preference Optimization: Mechanisms, Consequences, and Mitigation via Tie Training.TL;DROur theorems and experiments suggest that DPO and RLHF have an unwelcome consequence: they make…
We need 3rd party Training-Run Assessments
via LessWrong AI [8] — Training-run assessments conducted by a 3rd party should become a standard part of frontier AI safety.By a Training-Run Assessment, or TRA, I mean an in-depth analysis of the post-training pipeline and dynamics leading up to a frontier model release. A TRA…
Infuriating Google commercial imagines the founding fathers embracing AI
via The Verge AI [4] — "Group project, but make it 1776." That's how a new commercial for Google Workspace opens. And things only get cringier from there. The clip imagines what it would be like if the founding fathers turned to Google's collaboration tools and Gemini to help…
Pragmatic FDT, and predictors as game theory
via Alignment Forum [999] — Decision theory is back in fashion (defining fashion as "one good post on a good EA blog"). Bentham's Bulldog (BB) has published a case against FDT (functional decision theory), contrasting rationalist enthusiasm with academic scepticism: "Academic…
Fable #6: The Return of the King
via Substack Zvi [999] — The blip is over.
A behind-the-scenes look at Midjourney’s medical scanner leaves many questions unanswered
via The Verge AI [4] — Midjourney has shown more of its futuristic medical scanner. It still hasn't shown much proof it works. The AI startup, best known for generating images, released a behind-the-scenes video of its dunk-tank ultrasound scanner, which it plans to deploy in…
AI Futurism Reading List
via LessWrong AI [5] — We at Redwood recently ran a strategy fellowship through Astra. As part of this, we ran a reading group for our fellows on some of the topics that we think are important for thinking about AI futurism (key dynamics in AI development, existential risk from…
The AFFINE Superintelligence Alignment Seminar – A Retrospective
via LessWrong AI [4] — A Day at AFFINE[1]“AFFINE was the best month of intellectual exploration I have had the opportunity to engage in, ever. Usually opportunities like this are limited to a day or a weekend, which both limits depth, forces a sprint-type mindset, and generally…
AI #175: The Fable Continues
via Substack Zvi [999] — Fable’s back.
Teaching AI to run with the turbines
via MIT Technology Review [4] — Artificial intelligence may have captured the public imagination through chatbots and image generators, but some of its most consequential use cases are unfolding far from consumer-facing tools. In industries where physical infrastructure,…
When capabilities work is the *safe* bet
via LessWrong AI [5] — If you believe that LLMs lend themselves unusually well to alignment compared to other regimes, this can be a very good reason to start doing capability research on them rather than LLM safety research. Imagine you have these beliefs about how AI goes:By I…
Why California’s carbon manure math doesn’t add up
via MIT Technology Review [4] — Something stinks in California’s climate policies. Years ago, the state set up a system that pays cattle farmers across the country to turn the methane emitted from cattle manure into natural gas, encouraging the dairy sector to produce a gas we…
Constructive Alignment: Governing Preference Dynamics in Human-AI Interaction
via ArXiv cs.AI [5] — Most approaches to AI alignment treat human preferences as fixed targets to be inferred and optimized. This assumption conflicts with extensive empirical evidence showing that preferences are layered, dynamic, and constructed through…
Live Doom Meter
--
%
0% — We're fine
100% — GG
The Doom Meter is a composite score derived from prediction markets and feed sentiment, updated daily.
70%
Prediction Markets
Weighted average of Manifold Markets questions on AI catastrophe, AGI timelines, expert surveys, and key figures. Direct doom indicators weighted higher than indirect capability markers.
30%
Feed Sentiment
Percentage of recent headlines containing high-alarm keywords (existential risk, catastrophe, extinction). Higher alarm density = higher score.
This is not a scientific estimate of existential risk. It is an opinionated, transparent signal — a vibes-based thermometer for AI doom discourse.
P(Doom) Scoreboard
0%25%50%75%100%
Loading estimates...
Recent Voices
We are creating something that will be more powerful than us. I don't know a good precedent for a less intelligent thing managing a more intelligent thing.
— Geoffrey Hinton, Nobel Prize Lecture, Dec 2024
If you're not worried about AI safety, you're not paying attention.
— Sen. Blumenthal, Senate AI Hearing, 2024
The probability of doom is high enough that we should be working very hard to reduce it.
— Yoshua Bengio, MILA Talk, 2024