via LessWrong AI [6] — This is the fifth post in the sequence Implications of Continual Learning for LLM Agents.SummaryWhile writing our continual learning sequence, we sent a survey to a number of AI safety researchers with questions about continual learning. This post…

via LessWrong AI [9] — A notable fraction of people respond to hearing about existential risk from AI by saying they don’t really care if everyone dies. I think the idea is often along the lines of ‘well if we are all dead, then there’s nobody to be unhappy about it’.I’m…

via LessWrong AI [5] — I was chatting with someone tonight about a planned documentary; they had interviewed various people in AI safety, and we got to discussing who they should talk to from an e/acc (effective accelerationist) perspective. I also watched The AI Doc recently,…

via LessWrong AI [5] — I often hear people say they think we should pause AI at some point, but not yet. Their basis for this seems to be some combination of:If we pause at the last possible moment, then we will have the most advanced AI possible during the pause, which will be…

via LessWrong AI [3] — This work was done as part of the MATS fellowship by Joey Yudelson and Vladimir Ivanov. It was mentored by Ryan Greenblatt. Thanks to Aghyad Deeb and Anders Woodruff for comments on this post. Thanks to Monte MacDiarmid, Evan Hubinger, Sid Black, Satvik…

via Substack Zvi [999] — It does look good, actually.

via LessWrong AI [4] — In the game "The choice before us" by Nick Shapiro,[1] you are put in the shoes of an AI company leader. You grow your business. You unlock "wonders", such as curing cancer. All the while, you're attempting to avoid your product getting smart enough to…

via LessWrong AI [4] — Crosspost of my substack piece, covering quick thoughts on AI overcoming nuclear deterrence. TLDR: Nuclear deterrents likely only buy time to further invest in more resilient second-strike guarantees: without a comparable AI base, this will not happen fast…

via LessWrong AI [4] — For a while there, many people thought vitamin D was magical—that it could improve bones, the heart, infections, cancer, heart disease, longevity, even mental health. But among people I respect, opinion is now overwhelmingly that taking vitamin D does…

via Substack Zvi [999] — Your monthly hit of all the things that are fit to print without a better place to live.

via LessWrong AI [4] — In Parts 1, 2, and 3 we estimated how fast a post-AGI economy could grow using existing or historically observed production techniques, grounded in US input-output data. That approach gave us confidence that the methods we assumed were physically…

via Substack Zvi [999] — GLM-5.2 arrived last week.

via LessWrong AI [5] — Here’s Holden Karnofsky:I tend to think it’s worse than 51/49. I tend to think we’re always going to be prone to overestimate how robustly good our actions are. And the more we learn about all the galaxy-brained considerations that one should have had in…

via LessWrong AI [3] — Tldr: Most strategic writing on AI governance on LessWrong describes the outsider game, which is most often visible: press, statements, open letters. Here I want to describe the other, invisible half: the insider work within ministerial cabinets and…

via LessWrong AI [5] — These are some personal notes taken and later dressed up a bit to make into a post. Dunno how much value is here for people already familiar with the AI Safety Ecosystem.Over several weeks in the spring of 2026 I attempted to map out the entire AI Safety…

via LessWrong AI [4] — Short introductory post for my research direction: Gaussian Natural Latents. I explain the motivation and give a preview of the forthcoming results.The Natural Abstractions agenda, in my view, is a promising research program that asks important theoretical…

via Substack Zvi [999] — Only three days after the release of Claude Fable 5, Anthropic was forced by the United States Government to make it unavailable, when a jailbreak was brought to its attention, rather than the previous situation of ‘yes obviously experts can jailbreak…

via LessWrong AI [5] — This post was written while working for Arcadia Impact's Alignment Team (and grew out of an internal talk I gave) but is my own opinion and not theirs. I am grateful for feedback from Daniel Tan and the rest of the team.This post was originally going to be…

via LessWrong AI [7] — Context: We are the ‘model motivations’ team at Arcadia Alignment. We aim to build a science of ‘model intentions’, unifying insights from personas and other empirical evidence. In this post, we’ll outline the need for much better model organisms and how…

via LessWrong AI [5] — From the end of high school to after my sophomore year of college, I considered myself an effective altruist. I was on the board of my college EA club, ran an EA intro fellowship, and went to EA retreats. I was vegetarian, regularly donated to GiveWell,…