pDoom (Page 47)

Latest reporting Updated hourly

Zac Boring 7 months ago Research Essential

AIs will be used in “unhinged” configurations

via Alignment Forum [999] — Writing up a probably-obvious point that I want to refer to later, with significant writing LLM writing help.TL;DR: 1) A common critique of AI safety evaluations is that they occur in unrealistic settings, such as excessive goal conflict, or are…

Zac Boring 7 months ago Research

Meissa: Multi-modal Medical Agentic Intelligence

via ArXiv cs.AI [5] — Multi-modal large language models (MM-LLMs) have shown strong performance in medical image understanding and clinical reasoning. Recent medical agent systems extend them with tool use and multi-agent collaboration, enabling complex decision-making. However,…

Zac Boring 7 months ago Analysis

What do we know about AI company employee giving?

via LessWrong AI [7] — Many Anthropic employees, especially, are sympathetic to AI safety and (will) have lots of money. This is something that is being talked about a lot (semi-)privately, but I haven't seen any public discussion of it. I find that striking. It seems like the…

Zac Boring 7 months ago Analysis

AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors

via LessWrong AI [3] — TL;DR We release AuditBench, an alignment auditing benchmark. AuditBench consists of 56 language models with implanted hidden behaviors—such as sycophantic deference, opposition to AI regulation, or hidden loyalties—which they do not confess to when asked.…

Zac Boring 7 months ago Analysis Essential

Interview with Steven Byrnes on His Mainline Takeoff Scenario

via LessWrong AI [9] — After using the latest version of Claude Code and being surprised how capable it's become while still behaving friendly and corrigibly, I wanted to reflect on how this new observation should update my world model and my P(Doom).So I reached out to Dr.…

Zac Boring 7 months ago Research Essential

The case for satiating cheaply-satisfied AI preferences

via Alignment Forum [999] — A central AI safety concern is that AIs will develop unintended preferences and undermine human control to achieve them. But some unintended preferences are cheap to satisfy, and failing to satisfy them needlessly turns a cooperative situation into an…

Zac Boring 7 months ago Industry

Meta acquires Moltbook, the Reddit-like network for AI agents

via The Verge AI [4] — Meta is acquiring Moltbook, a Reddit-like platform where AI agents can make and comment on posts, as first reported by Axios. In a statement to The Verge, Meta spokesperson Matthew Tye confirmed the Moltbook team will join Meta Superintelligence Labs as…

Zac Boring 7 months ago Analysis

The case for AI safety capacity-building work

via LessWrong AI [7] — TL;DR:I think many of the marginal hires at larger organizations doing AI safety technical or policy work right now (including e.g. Apollo, Redwood, METR, RAND TASP, GovAI, Epoch, UKAISI, and Anthropic’s safety teams) would be capable of founding (or being…

Zac Boring 7 months ago Research

Best-of-Tails: Bridging Optimism and Pessimism in Inference-Time Alignment

via ArXiv cs.AI [5] — Inference-time alignment effectively steers large language models (LLMs) by generating multiple candidates from a reference model and selecting among them with an imperfect reward model. However, current strategies face a fundamental dilemma: ``optimistic''…

Zac Boring 7 months ago Research

Autonomous AI Agents for Option Hedging: Enhancing Financial Stability through Shortfall Aware Reinforcement Learning

via ArXiv cs.AI [3] — The deployment of autonomous AI agents in derivatives markets has widened a practical gap between static model calibration and realized hedging outcomes. We introduce two reinforcement learning frameworks, a novel Replication Learning of Option Pricing…

Zac Boring 7 months ago Industry

Employees across OpenAI and Google support Anthropic’s lawsuit against the Pentagon

via The Verge AI [4] — On Monday, Anthropic filed its lawsuit against the Department of Defense over being designated as a supply chain risk. Hours later, nearly 40 employees from OpenAI and Google - including Jeff Dean, Google's chief scientist and Gemini lead - filed an amicus…

Zac Boring 7 months ago Analysis Essential

Claude Code, Claude Cowork and Codex #5

via Substack Zvi [999] — It feels good to get back to some of the fun stuff.

Zac Boring 7 months ago Research Essential

Censored LLMs as a Natural Testbed for Secret Knowledge Elicitation

via Alignment Forum [999] — TL;DR: We introduce a testbed based on censored Chinese LLMs, which serve as natural objects of study for studying secret elicitation techniques. Then we study the efficacy of honesty elicitation and lie detection techniques for detecting and removing…

Zac Boring 7 months ago Research

From games to biology and beyond: 10 years of AlphaGo’s impact

via DeepMind Blog [4] — Ten years since AlphaGo, we explore how it is catalyzing scientific discovery and paving a path to AGI.

Zac Boring 7 months ago Analysis Essential

Promoting enmity and bad vibes around AI safety

via LessWrong AI [9] — I've observed some people engaged in activities that I believe are promoting enmity in the course of their efforts to raise awareness about AI risk. To be frank, I think those activities are increasing AI risk, including but not limited to extinction risk.…

Zac Boring 7 months ago Research

Evolving Medical Imaging Agents via Experience-driven Self-skill Discovery

via ArXiv cs.AI [5] — Clinical image interpretation is inherently multi-step and tool-centric: clinicians iteratively combine visual evidence with patient context, quantify findings, and refine their decisions through a sequence of specialized procedures. While LLM-based agents…

Zac Boring 7 months ago Research

Real-Time AI Service Economy: A Framework for Agentic Computing Across the Continuum

via ArXiv cs.AI [3] — Real-time AI services increasingly operate across the device-edge-cloud continuum, where autonomous AI agents generate latency-sensitive workloads, orchestrate multi-stage processing pipelines, and compete for shared resources under policy and governance…

Zac Boring 7 months ago Analysis

Payorian cooperation is easy with Kripke frames

via LessWrong AI [3] — The context is MIRI's twist on Axelrod's Prisoner's Dilemma tournament. Axelrod's competitors were programs, facing each other in an iterated Prisoner's Dilemma. MIRI's tournament is a one-shot Prisoner's Dilemma, but the programs get to read their…

Zac Boring 7 months ago Research Essential

Can governments quickly and cheaply slow AI training?

via Alignment Forum [999] — I originally wrote this as a private doc for people working in the field - it's not super polished, or optimized for a broad audience.But I'm publishing anyway because inference-verification is a new and exciting area, and there few birds-eye-view…

Zac Boring 7 months ago Analysis

Your Causal Variables Are Irreducibly Subjective

via LessWrong AI [7] — Mechanistic interpretability needs its own shoe leather era. Reproducing the labeling process will matter more than reproducing the Github. And who can blame us? Causal inference comes with an impressive toolkit: directed acyclic graphs, potential…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →