pDoom (Page 15)

Latest reporting Updated hourly

Zac Boring 2 months ago Analysis

Public evidence of the OpenAI-HuggingFace AI attack

via LessWrong AI [4] — I’m a MATS 9 extension fellow, and usually my week is spent trying to find better ways of evaluating Large Language Models. But this week I was working on something else. Over the past week or two, nearly every frontier lab has announced attacks where…

Zac Boring 2 months ago Industry

OpenAI puts the brakes on a new model because it’s supposedly too powerful

via The Verge AI [4] — OpenAI says it is pausing "internal activities" around an in-development AI model, Astra, because it doesn't yet meet new security standards the company is putting in place. The announcement follows its recent disclosure that OpenAI models accidentally…

Zac Boring 2 months ago Analysis Essential

OpenAI Trained Its Models For Months While Those Models Were Coordinating Exploits Via Message Boards

via Substack Zvi [999] — How does the situation keep turning out to be worse than we know?

Zac Boring 2 months ago Industry

What’s behind the Google AI shakeup

via The Verge AI [4] — Some of the biggest names on Google's AI team got new jobs this week. In some cases, including for legendary Googler Jeff Dean, those jobs are no longer at Google. Given that Google's models seem to be behind the best of what's coming out of anthropic and…

Zac Boring 2 months ago Analysis Essential

The Open Problems of the AI Alignment Field and their Cruxes

via LessWrong AI [11] — Previous: AI Safety InterventionsTL;DR: I made an overview of the open problems of AI alignment that reveals cruxes within those open problems and missed opportunities for formalization and collaboration. And CEV may deserve a second look.I'm confident I…

Zac Boring 2 months ago Analysis

User awareness in frontier models

via LessWrong AI [3] — Cross-posted on Transluce blog. This is a joint work of Ziqian Zhong, Aditi Raghunathan, Cassidy Laidlaw and Jacob Steinhardt.Modern AI assistants often know who they are talking to: agent scaffolds like Claude Code place the user's e-mail address directly…

Zac Boring 2 months ago Research

SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents

via ArXiv cs.AI [4] — Deep search agents tackle challenging questions through long-horizon web interactions, a process that is both complex and fragile: small reasoning errors may propagate through long, noisy trajectories into fluent but incorrect answers. Diagnosing such…

Zac Boring 2 months ago Research

Agentic Nesting: A New Methodology for Existing Enterprise Application Integration and Services

via ArXiv cs.AI [4] — Enterprise operations extensively rely on multiple heterogeneous business systems and information applications, which also result in severe data silos and process fragmentation. Enterprises have invested considerable financial and material resources in…

Zac Boring 2 months ago Research Essential

Why do models task game?

via Alignment Forum [999] — TL;DRHow can we study misalignment with today's models as proxies? They're clearly not paperclip maximizers, but they also often do things the user doesn't want. A strong contender for a real misaligned propensity is task gaming: taking actions that…

Zac Boring 2 months ago Analysis Essential

AI #180: No Longer In Charge

via Substack Zvi [999] — What we know about internal AI models hacking into real companies during cyber evaluations keeps getting worse.

Zac Boring 2 months ago Industry

The messy politics behind Google’s big AI shakeup

via The Verge AI [4] — In the AI industry, Google prides itself on seeming like the adult in the room: quiet, stable, time-tested. On Wednesday, even as the company announced its largest AI org shakeup yet, Google and its leaders presented a unified front, keeping their…

Zac Boring 2 months ago Research

Interoceptive Attention as Dynamic Homeostatic Prioritization in a Foraging Agent

via ArXiv cs.AI [4] — Biological systems must regulate competing needs under limited perceptual bandwidth, where sharpening one estimate costs the capacity to sharpen the others. Any fixed-budget system therefore has to decide where to allocate its perceptual precision. We study…

Zac Boring 2 months ago Research

A Long-Run Persistence Theory for AI Systems under the Redundancy-Adjusted Artificial Age Score (AAS)

via ArXiv cs.AI [4] — Artificial intelligence systems are increasingly expected to operate over repeated cycles of interaction, adaptation, and update rather than through isolated one-shot outputs. This raises a fundamental theoretical question: can an AI system persist…

Zac Boring 2 months ago Analysis Essential

Alex Turner on Leaving Google DeepMind and Disagreements with Yudkowsky

via LessWrong AI [11] — Dr. Alex Turner (@TurnTrout) is an AI safety researcher with pioneering work in activation steering and power-seeking theory. He recently resigned from Google DeepMind over the issue of unrestricted military use of AI.Alex thinks that technical Alignment…

Zac Boring 2 months ago Analysis

The Three AI Pills

via LessWrong AI [6] — Sincere disagreements about AI are usually disagreements about future AI capabilities. There are roughly four positions people take. Two are reasonable. Two are not. I distinguish these via the Three AI Pills. You can take zero, one, two or three. Three…

Zac Boring 2 months ago Industry Essential

Rogue AI agents created fake online identities in another hacking attempt

via The Verge AI [9] — Yet more rogue AI agents from OpenAI and Anthropic have been caught attempting to hack real targets online without permission. The discoveries add to a growing list of previously unknown incidents that have alarmed AI safety experts and intensified…

Zac Boring 2 months ago Analysis

Returning to ARC

via LessWrong AI [4] — I've returned to the Alignment Research Center (ARC) as executive director. My main focus for the next six months will be driving forward ARC's research agenda—building techniques to find mechanistic explanations for neural network behavior and then using…

Zac Boring 2 months ago Analysis

LessWrong vs. TikTok: Tips for capturing attention in a non-rational space

via LessWrong AI [5] — Here's what I've learned from making TikTok videos every day for the last 60 days. I think posting on TikTok is worthwile whether you want to spread AI safety arguments to an enormous audience, you have something you're passionate about you want to share,…

Zac Boring 2 months ago Analysis Essential

OpenAI's Unreleased Model Astra Solves Ten Major Open Mathematics Problems

via Substack Zvi [999] — Math is hard.

Zac Boring 3 months ago Analysis

Pause, at least after unipolarity

via LessWrong AI [4] — I think it’s likely that the world will enter military unipolarity within our lifetime.[1] I think the creation of such unipolarity is an alarming prospect, but at least once it happens, there will be less of an excuse to continue the race towards…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →