via LessWrong AI [4] — Lately I've been thinking a lot about what work would help with actually winning and getting to good worlds. In the spirit of that I decided to venture outside my normal wheelhouse and spend some time reflecting on what technical research could make me…

via Substack Zvi [999] — We, or at least ‘more than 100 American institutions,’ got Mythos back this week.

via LessWrong AI [7] — p(doom) is a shorthand for some important bits and a way to notice a disagreement to double-crux about.If you work on AI capabilities at a frontier AI company, I might ask you for your p(doom).If it's less than 1%, I know that you're probably not familiar…

via LessWrong AI [4] — tl;dr: As recursive self-improvement accelerates, we need a top-level agenda to research how to effectively keep humans in the loop. We need to study how humans can best interpret and guide research performed by autonomous agents when those agents lack…

via LessWrong AI [4] — An argument goes: If we don't build aligned artificial superintelligence, we risk driving ourselves extinct for some other reason. We should rush to build ASI quickly, in spite of the risks—the longer we wait, the more vulnerable we are to extinction from…

via LessWrong AI [5] — TLDR: what is the grant round?grantmaking.ai is launching a $1M grant round, distributing $5k to $50k per successful application to people and projects working to reduce x-risk from AI.Applications will be reviewed by Gavin Leech, Ryan Kidd, and Marcus…

via LessWrong AI [10] — Look, I'm as much of a Rationalist with a special interest in AI x-risk as anyone. But oh my god do I hate talking about "P(doom)". When it first started showing up in the wake of ChatGPT, I assumed that it was floating around variously adjacent circles…

via Substack Zvi [999] — The Wall Street Journal printed an outright false headline and heavily misleading story claiming this, which of course was uncritically amplified by the usual suspects.

via LessWrong AI [5] — I, along with many others in AI safety, believe there is a shortage of generalists in the community and that there exist many projects and efforts that by default will not happen unless they are owned by a strong generalist[1][2][3]. As someone who is a…

via Substack Zvi [999] — While we wait for a general release, the system card is the best hint as to what is going on with the new candidate for America’s Next Top Model, GPT-5.6.

via LessWrong AI [5] — @habryka and I recently spoke about his plans to improve the AI safety funding ecosystem with a better S-Process platform, and my new incubator for EA/AIS software projects, Surplus (since launched; apply now!)We also cover: hot takes on different funders;…

via LessWrong AI [4] — When I talk to people about what might be done about AI threatening approximately everything that everyone cares about, I notice a common oddity in their resistance to a variety of ideas. They seem to take for granted that certain entities—especially Trump…

via LessWrong AI [5] — This was too long to be a short-form, but it should really be a short-form.This notice is useful for people who've recently got into AI safety, who want to engage with the ancient texts (i.e. pre-2024). If you were around before 2023, then you probably…

via LessWrong AI [4] — Car crashes kill over 35,000 people in the US every year. Plane crashes, on the other hand, kill ~350. Despite this, we have shows like Mayday/Air Disasters for entertainment on TV, and events such as the tragic death of 67 people on a commercial airline…

via LessWrong AI [10] — The existential AI safety community needs to take building a civic and social movement seriously as a core intervention. We believe this is a high-value, badly neglected approach to reducing catastrophic/x-risks from AI because it may significantly…

via Substack Zvi [999] — We have a new standard policy for releasing frontier AI models. It is not good.

via LessWrong AI [4] — 1. The obstacle to abolition was not the economic system, but an industry lobby.I had always imagined the British abolitionist movement to be a broad battle between an unstoppable moral imperative and an immovable economic incentive. But in practice it…

via LessWrong AI [4] — Imagine eleven people each have a message that they think should get 10% of a group’s attention. They aren’t being crazy selfish and attention-seeking - just on average a little over-emphasizing the importance of their own information. So adding up all the…

via Substack Zvi [999] — Fable remains in limbo, with renewed hope that we will get it back soon (45% by tomorrow, 69% by July 1, nice.) The full capabilities post is now available.

via LessWrong AI [6] — This is the fifth post in the sequence Implications of Continual Learning for LLM Agents.SummaryWhile writing our continual learning sequence, we sent a survey to a number of AI safety researchers with questions about continual learning. This post…