Zac Boring 4 months ago Analysis Essential GPT-5.6: The System Card via Substack Zvi [999] — While we wait for a general release, the system card is the best hint as to what is going on with the new candidate for America’s Next Top Model, GPT-5.6. Save Read
Zac Boring 4 months ago Analysis Austin & Oli on funding and incubating projects via LessWrong AI [5] — @habryka and I recently spoke about his plans to improve the AI safety funding ecosystem with a better S-Process platform, and my new incubator for EA/AIS software projects, Surplus (since launched; apply now!)We also cover: hot takes on different funders;… Save Read
Zac Boring 4 months ago Analysis Why are adversaries assumed to be incapable of responding to AI risk? via LessWrong AI [4] — When I talk to people about what might be done about AI threatening approximately everything that everyone cares about, I notice a common oddity in their resistance to a variety of ideas. They seem to take for granted that certain entities—especially Trump… Save Read
Zac Boring 4 months ago Analysis What did "scheming" and "mech interp" mean pre-2023? via LessWrong AI [5] — This was too long to be a short-form, but it should really be a short-form.This notice is useful for people who've recently got into AI safety, who want to engage with the ancient texts (i.e. pre-2024). If you were around before 2023, then you probably… Save Read
Zac Boring 4 months ago Research Essential Deployment Awareness Matters More Than Evaluation Awareness via Alignment Forum [999] — TL;DREvaluation awareness — an AI recognizing it's being evaluated — is a widely discussed concept in AI safety. But there is a closely related concept that we claim is more important: deployment awareness, the AI's ability to recognize when it is not… Save Read
Zac Boring 4 months ago Analysis Don't ignore the car crashes, and remember your freshman CS via LessWrong AI [4] — Car crashes kill over 35,000 people in the US every year. Plane crashes, on the other hand, kill ~350. Despite this, we have shows like Mayday/Air Disasters for entertainment on TV, and events such as the tragic death of 67 people on a commercial airline… Save Read
Zac Boring 4 months ago Analysis Essential Existential AI safety needs an effective social movement. PauseAI is building it via LessWrong AI [10] — The existential AI safety community needs to take building a civic and social movement seriously as a core intervention. We believe this is a high-value, badly neglected approach to reducing catastrophic/x-risks from AI because it may significantly… Save Read
Zac Boring 4 months ago Research Essential The Case for Model Forensics via Alignment Forum [999] — If we had a misalignment warning shot, would we be able to tell?Suppose an AI company catches their model taking an egregious action, like deleting oversight code that monitors its actions. Should they sound the alarm? A key piece of evidence to… Save Read
Zac Boring 4 months ago Analysis Essential White House Will Ad Hoc Decide Who Can Individually Access GPT-5.6 via Substack Zvi [999] — We have a new standard policy for releasing frontier AI models. It is not good. Save Read
Zac Boring 4 months ago Analysis Surprising facts about the slave trade via LessWrong AI [4] — 1. The obstacle to abolition was not the economic system, but an industry lobby.I had always imagined the British abolitionist movement to be a broad battle between an unstoppable moral imperative and an immovable economic incentive. But in practice it… Save Read
Zac Boring 4 months ago Research Governing Actions, Not Agents: Institutional Attestation as a Governance Model for Autonomous AI Systems via ArXiv cs.AI [3] — Autonomous AI agents may begin to perform consequential, irreversible actions such as clinical prescribing and production software deployment. This paper observes that human institutions have governed powerful autonomous actors not by monitoring their… Save Read
Zac Boring 4 months ago Research Detecting and Controlling Sycophancy with Cascading Linear Features via ArXiv cs.AI [3] — Interpreting and controlling model behaviors through activation steering methods requires many pairs of contrastive samples that clearly exhibit desired or undesired behavior. These data pairs determine the degree to which interpretability frameworks can… Save Read
Zac Boring 4 months ago Analysis The shouting equilibrium via LessWrong AI [4] — Imagine eleven people each have a message that they think should get 10% of a group’s attention. They aren’t being crazy selfish and attention-seeking - just on average a little over-emphasizing the importance of their own information. So adding up all the… Save Read
Zac Boring 4 months ago Analysis Essential AI #174: You're It via Substack Zvi [999] — Fable remains in limbo, with renewed hope that we will get it back soon (45% by tomorrow, 69% by July 1, nice.) The full capabilities post is now available. Save Read
Zac Boring 4 months ago Industry Facebook’s Creator Studio has been revived as an AI companion app via The Verge AI [4] — Meta is bringing back the Facebook Creator Studio page manager, now "reimagined" as a standalone AI companion app. The new app aims to make it easier for creators to connect with their audiences and show them "exactly how to grow on Facebook," according to… Save Read
Zac Boring 4 months ago Research The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing via ArXiv cs.AI [3] — Autonomous AI systems are transitioning from advisory to autonomous roles for medication prescriptions. Recent United States bill H.R. 238 and Utah's prescription-renewal pilot both authorize AI to prescribe medications in an agentic capacity. While some… Save Read
Zac Boring 4 months ago Research The Hitchhiker's Guide to Agentic AI: From Foundations to Systems via ArXiv cs.AI [4] — The Hitchhiker's Guide to Agentic AI is a comprehensive practitioner's reference for building autonomous AI systems. The book covers the full stack from first principles to production deployment, organized around a central thesis: building great agentic… Save Read
Zac Boring 4 months ago Analysis Perspectives on Continual Learning: Survey Results and Forecasts via LessWrong AI [6] — This is the fifth post in the sequence Implications of Continual Learning for LLM Agents.SummaryWhile writing our continual learning sequence, we sent a survey to a number of AI safety researchers with questions about continual learning. This post… Save Read
Zac Boring 4 months ago Analysis Essential AI catastrophe: more like a genocide than a thought experiment via LessWrong AI [9] — A notable fraction of people respond to hearing about existential risk from AI by saying they don’t really care if everyone dies. I think the idea is often along the lines of ‘well if we are all dead, then there’s nobody to be unhappy about it’.I’m… Save Read
Zac Boring 4 months ago Analysis What is up with e/acc? via LessWrong AI [5] — I was chatting with someone tonight about a planned documentary; they had interviewed various people in AI safety, and we got to discussing who they should talk to from an e/acc (effective accelerationist) perspective. I also watched The AI Doc recently,… Save Read