pDoom (Page 36)

Latest reporting Updated hourly

Zac Boring 5 months ago Analysis

AI Industrial Takeoff — Part 1: Maximum growth rates with current technology

via LessWrong AI [4] — How fast could an AI-driven economy grow? Most economists expect a few percentage points at best, comparable to previous general-purpose technologies (Acemoglu (2024)). Those closer to AI development tend to imagine something much more radical (Shulman…

Zac Boring 5 months ago Industry

Tailoring AI solutions for health care needs

via MIT Technology Review [4] — The AI market is full of big promises of grand transformation. Health care is a prime target for those promises, beset as it is by financial pressures, labor shortages, and the growing burden of caring for an aging population. AI developers are…

Zac Boring 6 months ago Research Essential

TUR-DPO: Topology- and Uncertainty-Aware Direct Preference Optimization

via ArXiv cs.AI [9] — Aligning large language models (LLMs) with human preferences is commonly done via reinforcement learning from human feedback (RLHF) with Proximal Policy Optimization (PPO) or, more simply, via Direct Preference Optimization (DPO). While DPO is stable and…

Zac Boring 6 months ago Research

Minimal, Local, Causal Explanations for Jailbreak Success in Large Language Models

via ArXiv cs.AI [5] — Safety trained large language models (LLMs) can often be induced to answer harmful requests through jailbreak prompts. Because we lack a robust understanding of why LLMs are susceptible to jailbreaks, future frontier models operating more autonomously in…

Zac Boring 6 months ago Analysis

OpenAI's red line for AI self-improvement is fundamentally flawed

via LessWrong AI [4] — TL;DR. OpenAI's "Critical" threshold for AI self-improvement in the Preparedness Framework v2 has three structural problems:It fires too late. The lagging indicator, 5× generational acceleration sustained for several months, lets ~3 years of effective…

Zac Boring 6 months ago Research Essential

Exploration Hacking: Can LLMs Learn to Resist RL Training?

via Alignment Forum [999] — We empirically investigate exploration hacking (EH) — where models strategically alter their exploration to resist RL training — by creating model organisms that resist capability elicitation, evaluating countermeasures, and auditing frontier models…

Zac Boring 6 months ago Research Essential

Risk from fitness-seeking AIs: mechanisms and mitigations

via Alignment Forum [999] — Current AIs routinely take unintended actions to score well on tasks: hardcoding test cases, training on the test set, downplaying issues, etc. This misalignment is still somewhat incoherent, but it increasingly resembles what I call…

Zac Boring 6 months ago Analysis Essential

Housing Roundup #14: You Can't Build That

via Substack Zvi [999] — Why can’t you build it?

Zac Boring 6 months ago Analysis Essential

AI unemployment and AI extinction are often the same

via LessWrong AI [10] — My sense is that people think of AI existential risk and AI unemployment as distinct issues. Some people are extremely concerned about extinction and perhaps even indifferent to total unemployment. Some people think of moderate AI unemployment as a…

Zac Boring 6 months ago Industry

Pentagon strikes classified AI deals with OpenAI, Google, and Nvidia — but not Anthropic

via The Verge AI [4] — The Pentagon has struck deals with OpenAI, Google, Microsoft, Amazon, Nvidia, Elon Musk's xAI, and the startup Reflection, allowing the agency to use their AI tools in classified settings, according to an announcement on Friday. At the same time, the…

Zac Boring 6 months ago Industry

Microsoft wants lawyers to trust its new AI agent in Word documents

via The Verge AI [4] — Microsoft is launching a new AI agent inside Word that's specifically designed for legal teams. Legal Agent handles document edits, negotiation history, and complex documents to help legal teams handle tasks like reviewing contracts. "Instead of relying on…

Zac Boring 6 months ago Analysis Essential

AI risk was not invented by AI CEOs to hype their companies

via LessWrong AI [9] — I hear that many people believe that the idea of advanced AI threatening human existence was invented by AI CEOs to hype their products. I’ve even been condescendingly informed of this, as if I am the one at risk of naively accepting AI companies’…

Zac Boring 6 months ago Research

Binary Spiking Neural Networks as Causal Models

via ArXiv cs.AI [4] — We provide a causal analysis of Binary Spiking Neural Networks (BSNNs) to explain their behavior. We formally define a BSNN and represent its spiking activity as a binary causal model. Thanks to this causal representation, we are able to explain the output…

Zac Boring 6 months ago Industry Essential

This startup’s new mechanistic interpretability tool lets you debug LLMs

via MIT Technology Review [8] — The San Francisco–based startup Goodfire just released a new tool, called Silico, that lets researchers and engineers peer inside an AI model and adjust its parameters—the settings that determine a model’s behavior—during training. This could give…

Zac Boring 6 months ago Analysis Essential

AI #166: Google Sells Out

via Substack Zvi [999] — This was the week of GPT-5.5.

Zac Boring 6 months ago Research

Distill-Belief: Closed-Loop Inverse Source Localization and Characterization in Physical Fields

via ArXiv cs.AI [3] — {Closed-loop inverse source localization and characterization (ISLC) requires a mobile agent to select measurements that localize sources and infer latent field parameters under strict time constraints.} {The core challenge lies in the belief-space…

Zac Boring 6 months ago Analysis

No Strong Orthogonality From Selection Pressure

via LessWrong AI [4] — A postratfic version of this essay, together with the acknowledgements for both, is available on SubstackEdit: if no one thinks an agent can become superintelligent and contest the lightcone while maintaining arbitrarily stupid goals, thats great! I’m only…

Zac Boring 6 months ago Research Essential

Research Sabotage in ML Codebases

via Alignment Forum [999] — One of the main hopes for AI safety is using AIs to automate AI safety research. However, if models are misaligned, then they may sabotage the safety research. For example, misaligned AIs may try to:Perform sloppy research in order to slow down the…

Zac Boring 6 months ago Industry

Building the compute infrastructure for the Intelligence Age

via OpenAI Blog [6] — OpenAI scales Stargate to build the compute infrastructure powering AGI, adding new data center capacity to meet growing AI demand.

Zac Boring 6 months ago Analysis Essential

The Most Important Charts In The World

via Substack Zvi [999] — We all need a break so: What is the most important chart in the world?

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →