pDoom (Page 43)

Latest reporting Updated hourly

Zac Boring 7 months ago Analysis

Nick Bostrom: How big is the cosmic endowment?

via LessWrong AI [4] — Superintelligence, pp. 122–3. 2014.Consider a technologically mature civilization capable of building sophisticated von Neumann probes of the kind discussed in the text. If these can travel at 50% of the speed of light, they can reach some stars before the…

Zac Boring 7 months ago Analysis

What if superintelligence is just weak?

via LessWrong AI [4] — In response to “2023 Or, Why I am Not a Doomer” by Dean W. Ball.Dean Ball is a pretty big voice in AI policy – over 19k subscribers on his newsletter, and a former Senior Policy Advisor for AI at the Trump White House – so why does he disagree that AI…

Zac Boring 7 months ago Research Essential

The AI Doc: Your Questions Answered

via MIRI [999] — So you’ve just seen The AI Doc, and you suddenly have questions, lots of them. The 104-minute documentary (currently in theaters) takes viewers on a fast-paced tour through the many dimensions of the AI problem, featuring interviews from a wide range of experts.…

Zac Boring 7 months ago Analysis

AI's capability improvements haven't come from it getting less affordable

via LessWrong AI [3] — METR's frontier time horizons are doubling every few months, providing substantial evidence that AI will soon be able to automate many tasks or even jobs. But per-task inference costs have also risen sharply, and automation requires AI labor to be…

Zac Boring 7 months ago Analysis

ControlAI 2025 Impact Report

via LessWrong AI [4] — This post highlights a few key excerpts from our full impact report. You can read the full report at https://controlai.com/impact-report-2025.ControlAI is a non-profit organization working to avert the extinction risks posed by superintelligence. We help…

Zac Boring 7 months ago Analysis Essential

Anthropic vs. DoW #6: The Court Rules

via Substack Zvi [999] — Last night, Anthropic was given its preliminary injunction, with a stay of seven days.

Zac Boring 7 months ago Research Essential

Trust as Monitoring: Evolutionary Dynamics of User Trust and AI Developer Behaviour

via ArXiv cs.AI [8] — AI safety is an increasingly urgent concern as the capabilities and adoption of AI systems grow. Existing evolutionary models of AI governance have primarily examined incentives for safe development and effective regulation, typically representing users'…

Zac Boring 7 months ago Research

ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence

via ArXiv cs.AI [4] — We introduce ARC-AGI-3, an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments in which agents must explore, infer goals, build internal models of environment dynamics, and plan effective action sequences…

Zac Boring 7 months ago Analysis

My hobby: running deranged surveys

via LessWrong AI [4] — In late 2024, I was on a long walk with some friends along the coast of the San Francisco Bay when the question arose of just how much of a bubble we live in. It’s well known that the Bay Area is a bubble, and that normal people don’t spend that much time…

Zac Boring 7 months ago Analysis Essential

Sen. Sanders (I-VT) and Rep. Ocasio-Cortez (D-NY) propose AI Data Center Moratorium Act

via LessWrong AI [15] — The text of the bill can be found here. It begins by citing the warnings of AI company CEOs and deep learning pioneers Geoffrey Hinton and Yoshua Bengio, the 2023 FLI open letter calling for a 6-month pause, and the 2025 FLI statement on…

Zac Boring 7 months ago Research Essential

Test your best methods on our hard CoT interp tasks

via Alignment Forum [999] — Authors: Daria Ivanova, Riya Tyagi, Arthur Conmy, Neel NandaDaria and Riya are co-first authors. This work was done during Neel Nanda’s MATS 9.0. Claude helped write code and suggest edits for this post.TL;DR One of our best safety techniques right…

Zac Boring 7 months ago Analysis Essential

AI #161 Part 1: 80,000 Interviews

via Substack Zvi [999] — The major technical advances this week were in agentic coding, as covered yesterday.

Zac Boring 7 months ago Research Essential

A Toy Environment For Exploring Reasoning About Reward

via Alignment Forum [999] — tldr: We share a toy environment that we found useful for understanding how reasoning changed over the course of capabilities-focused RL. Over the course of capabilities-focused RL, the model biases more strongly towards reward hints over direct…

Zac Boring 7 months ago Analysis

$1 billion is not enough; OpenAI Foundation must start spending tens of billions each year

via LessWrong AI [6] — OpenAI is now a public benefit corporation, with a charter that demands they use AGI for the benefit of all, and do so safely. To justify this structure to the Attorneys General of Delaware and California, they split off the nonprofit OpenAI Foundation,…

Zac Boring 7 months ago Analysis Essential

Claude Code, Cowork and Codex #6: Claude Code Auto Mode and Full Cowork Computer Use

via Substack Zvi [999] — Whatever else you think about Anthropic’s agentic coding department, they ship.

Zac Boring 7 months ago Industry

Agentic commerce runs on truth and context

via MIT Technology Review [4] — Imagine telling a digital agent, “Use my points and book a family trip to Italy. Keep it within budget, pick hotels we’ve liked before, and handle the details.” Instead of returning a list of links, the agent assembles an itinerary and executes…

Zac Boring 7 months ago Industry

The AI Hype Index: AI goes to war

via MIT Technology Review [4] — AI is at war. Anthropic and the Pentagon feuded over how to weaponize Anthropic’s AI model Claude; then OpenAI swept the Pentagon off its feet with an “opportunistic and sloppy” deal. Users quit ChatGPT in droves. People marched through London in…

Zac Boring 7 months ago Research

Intelligence Inertia: Physical Principles and Applications

via ArXiv cs.AI [3] — While Landauer's principle establishes the fundamental thermodynamic floor for information erasure and Fisher Information provides a metric for local curvature in parameter space, these classical frameworks function effectively only as approximations within…

Zac Boring 7 months ago Industry

Introducing the OpenAI Safety Bug Bounty program

via OpenAI Blog [7] — OpenAI launches a Safety Bug Bounty program to identify AI abuse and safety risks, including agentic vulnerabilities, prompt injection, and data exfiltration.

Zac Boring 7 months ago Analysis

The Fourth World

via LessWrong AI [4] — Is consciousness the last moral world?Imagine trying to explain to a virus why suffering matters.A virus is a simple self-replicating molecule: unsophisticated and arguably not even alive. It has no experience. It just copies itself according to chemical…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →