pDoom (Page 24)

Latest reporting Updated hourly

Zac Boring 4 months ago Analysis

Model access for third-parties — it's a big deal!

via LessWrong AI [5] — Over time, there might be an increasingly large gap between insider model access and outsider model access. By insiders, I mean employees at the frontier lab.[1] By "outsiders", I mean external safety researchers, third-party auditors, and other actors…

Zac Boring 4 months ago Research

What Drives Interactive Improvement from Feedback?

via ArXiv cs.AI [4] — We study when natural-language feedback produces improvement beyond the gains obtainable from repeated attempts alone. In multi-turn language agent setting, higher final accuracy can reflect useful feedback, but it can also arise from resampling, format…

Zac Boring 4 months ago Analysis

Structural Proxies

via LessWrong AI [4] — Lately I've been thinking a lot about what work would help with actually winning and getting to good worlds. In the spirit of that I decided to venture outside my normal wheelhouse and spend some time reflecting on what technical research could make me…

Zac Boring 4 months ago Analysis Essential

The Once And Future Fable #5

via Substack Zvi [999] — We, or at least ‘more than 100 American institutions,’ got Mythos back this week.

Zac Boring 4 months ago Research Essential

MIRI Newsletter #126

via MIRI [999] — Announcing: AI StopWatch In our last update, we mentioned we had something new in the works: a dedicated channel for news and analysis about AI. Subscribe to AI StopWatch An experiment from the writers and analysts at MIRI, AI StopWatch posts news and commentary…

Zac Boring 4 months ago Analysis

In partial defence of p(doom)

via LessWrong AI [7] — p(doom) is a shorthand for some important bits and a way to notice a disagreement to double-crux about.If you work on AI capabilities at a frontier AI company, I might ask you for your p(doom).If it's less than 1%, I know that you're probably not familiar…

Zac Boring 4 months ago Industry

Roundtables: Longevity’s Next Frontier: “Reprogramming” Your Body

via MIT Technology Review [4] — Listen to the session or watch below Billions of dollars are flooding into efforts to reverse aging as scientists explore ways to return cells to a younger state. But how far off are these experimental treatments? Will they really work? Watch a…

Zac Boring 4 months ago Research Essential

Summary: TGT’s 2026 ICML Papers

via MIRI [999] — The International Conference on Machine Learning (ICML), held annually for over forty years, is among the most influential conferences in modern AI research. This year in Seoul, ICML is hosting its second workshop on Technical AI Governance Research (TAIGR), and…

Zac Boring 4 months ago Industry

The Download: AI “coworkers” and stratospheric internet

via MIT Technology Review [4] — This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. AI agents are not your “coworkers” Imagine coming in to work to learn that a new underling will report to…

Zac Boring 4 months ago Analysis

Human-Guided Agentic Research: A Research Agenda

via LessWrong AI [4] — tl;dr: As recursive self-improvement accelerates, we need a top-level agenda to research how to effectively keep humans in the loop. We need to study how humans can best interpret and guide research performed by autonomous agents when those agents lack…

Zac Boring 4 months ago Industry

Building tech in the world’s secret R&D hub

via MIT Technology Review [4] — Apple. Anthropic. Disney Research. Google. Meta. Microsoft. NVIDIA. OpenAI. Few places outside Silicon Valley can claim R&D hubs from all of these companies. Fewer still are concentrated in a city of just over 400,000 people—roughly half the size…

Zac Boring 4 months ago Research

The Two Genie Game: Adoption and Welfare in Audit-Grounded AI Governance

via ArXiv cs.AI [6] — We ask under what conditions an agent with a harm-minimizing policy can displace an approval-seeking (RLHF) agent in a competitive market, and when that policy is sufficient to prevent community harm. We use evolutionary game theory (finite-population…

Zac Boring 4 months ago Research

IMCBench: A benchmark for multimodal LLMs in Image-grounded Medical Conversations

via ArXiv cs.AI [6] — Recent advances in large language models and vision-language models have enabled reasoning over multimodal data, offering opportunities for clinical applications such as decision support and triaging. However, existing medical AI benchmarks are fragmented:…

Zac Boring 4 months ago Analysis

AI will make biological extinction risks worse before it makes them better

via LessWrong AI [4] — An argument goes: If we don't build aligned artificial superintelligence, we risk driving ourselves extinct for some other reason. We should rush to build ASI quickly, in spite of the risks—the longer we wait, the more vulnerable we are to extinction from…

Zac Boring 4 months ago Industry

AI agents are not your “coworkers”

via MIT Technology Review [4] — This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here. Imagine coming in to work to learn that a new underling will report to you. The worker is not a person but an…

Zac Boring 4 months ago Analysis

$1M AI x-risk grant round is live on grantmaking.ai - apply for funding, review applicants, or fund projects

via LessWrong AI [5] — TLDR: what is the grant round?grantmaking.ai is launching a $1M grant round, distributing $5k to $50k per successful application to people and projects working to reduce x-risk from AI.Applications will be reviewed by Gavin Leech, Ryan Kidd, and Marcus…

Zac Boring 4 months ago Analysis Essential

P(doom) is a Dumb Meme

via LessWrong AI [10] — Look, I'm as much of a Rationalist with a special interest in AI x-risk as anyone. But oh my god do I hate talking about "P(doom)". When it first started showing up in the wake of ChatGPT, I assumed that it was floating around variously adjacent circles…

Zac Boring 4 months ago Analysis Essential

WSJ Article Claiming China Has Matched Anthropic Is Obvious Nonsense

via Substack Zvi [999] — The Wall Street Journal printed an outright false headline and heavily misleading story claiming this, which of course was uncritically amplified by the usual suspects.

Zac Boring 4 months ago Analysis

A reading list for generalists

via LessWrong AI [5] — I, along with many others in AI safety, believe there is a shortage of generalists in the community and that there exist many projects and efforts that by default will not happen unless they are owned by a strong generalist[1][2][3]. As someone who is a…

Zac Boring 4 months ago Industry

China’s Z.ai claims it can match Mythos on cybersecurity

via The Verge AI [4] — China's Zhipu AI (Z.ai) released its open-weight GLM-5.2, and some researchers have claimed that it matches Mythos in certain bug-finding and cybersecurity scenarios. While GLM lags behind models from Anthropic and OpenAI in other, more general tasks, it…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →