via LessWrong AI [3] — ProgramBench is a new coding benchmark that all frontier models spectacularly fail. We’ve been on a quest for “hard benchmarks” for a while so it’s refreshing to see a benchmark where top models do badly. Unfortunately, ProgramBench has one big problem:…

via Substack Zvi [999] — When I started this series, everyone was going crazy for coding agents.

via LessWrong AI [8] — TL;DR; CeSIA, the French Center for AI Safety is recruiting. French not necessary. Apply by 22 May 2026; Paris or remote in Europe/UK.On August 27, 2005, at an annual symposium in Jackson Hole, Raghuram Rajan, then chief economist of the International…

via Substack Zvi [999] — The era of training frontier models and then releasing them whenever you wanted?

via LessWrong AI [4] — I was thinking about Habryka's article on Putin's CEV, but I am posting my response here, because the original article is already 3 weeks old.I am not sure how exactly a person's CEV is defined. "If we knew everything and could self-modify" seems…

via LessWrong AI [5] — Sometimes, a friend who works around here, at an x-risk-themed organisation, will think about leaving their job. They’ll ask a group of people “what should I do instead?”. And everyone will chime in with ideas for other x-risk-themed orgs that they could…

via LessWrong AI [5] — SummaryLLMs are better at developing crystallized intelligence than fluid intelligence. That is: LLM training is good at building crystallized intelligence by learning patterns from training data, and this is sufficient to make them surprisingly skillful…

via Substack Zvi [999] — What is Anthropic?

via Substack Zvi [999] — The White House has ordered Anthropic not to expand access to Mythos, and is at least seriously considering a complete about-face of American Frontier AI policy into a full prior restraint regime, where anyone wishing to release a highly capable new…

via Substack Zvi [999] — So many are under the strange belief that there is something terrible about not owning the house in which you live.

via LessWrong AI [4] — How fast could an AI-driven economy grow? Most economists expect a few percentage points at best, comparable to previous general-purpose technologies (Acemoglu (2024)). Those closer to AI development tend to imagine something much more radical (Shulman…

via LessWrong AI [4] — TL;DR. OpenAI's "Critical" threshold for AI self-improvement in the Preparedness Framework v2 has three structural problems:It fires too late. The lagging indicator, 5× generational acceleration sustained for several months, lets ~3 years of effective…

via Substack Zvi [999] — Why can’t you build it?

via LessWrong AI [10] — My sense is that people think of AI existential risk and AI unemployment as distinct issues. Some people are extremely concerned about extinction and perhaps even indifferent to total unemployment. Some people think of moderate AI unemployment as a…

via LessWrong AI [9] — I hear that many people believe that the idea of advanced AI threatening human existence was invented by AI CEOs to hype their products. I’ve even been condescendingly informed of this, as if I am the one at risk of naively accepting AI companies’…

via Substack Zvi [999] — This was the week of GPT-5.5.

via LessWrong AI [4] — A postratfic version of this essay, together with the acknowledgements for both, is available on SubstackEdit: if no one thinks an agent can become superintelligent and contest the lightcone while maintaining arbitrarily stupid goals, thats great! I’m only…

via Substack Zvi [999] — We all need a break so: What is the most important chart in the world?

via Substack Zvi [999] — The system card for GPT-5.5 mostly told us what we expected.

via LessWrong AI [9] — A common thought pattern people seem to fall into when thinking about AI x-risk is approaching the problem as if the risk isn’t real, substantial, and imminent even if they think it is. When thinking this way, it becomes impossible to imagine the natural…