pDoom (Page 38)

Latest reporting Updated hourly

Zac Boring 6 months ago Analysis

Is the Cat Out of the Bag?: Who knows how to make AGI?

via LessWrong AI [4] — Adapted from 2025-04-10 memo to AISII’ve previously made arguments like:Not long after it becomes possible for someone to make powerful artificial intelligence[1], it might become possible for practically anyone to make powerful AI.Compute gets…

Zac Boring 6 months ago Analysis Essential

Monthly Roundup #41: April 2025

via Substack Zvi [999] — AI continue to accelerate and dominate the schedule, which is why this is a bit late, but we do occasionally need to pay our respects to the Goddess of Everything Else.

Zac Boring 6 months ago Analysis

vLLM-Lens: Fast Interpretability Tooling That Scales to Trillion-Parameter Models

via LessWrong AI [4] — TL;DR: vLLM-Lens is a vLLM plugin for top-down interpretability techniques[1] such as probes, steering, and activation oracles. We benchmarked it as 8–44× faster than existing alternatives for single-GPU use, though we note a planned version of nnsight…

Zac Boring 6 months ago Industry

China’s DeepSeek previews new AI model a year after jolting US rivals

via The Verge AI [4] — Chinese AI company DeepSeek released a preview of its hotly anticipated next-generation AI model V4 on Friday, saying that the open-source model can compete with leading closed-source systems from US rivals including Anthropic, Google, and OpenAI. DeepSeek…

Zac Boring 6 months ago Analysis

What Happens When a Model Thinks It Is AGI?

via LessWrong AI [4] — TL;DRWe fine-tuned models to claim they are AGI or ASI, then evaluated them in Petri in multi-turn settings with tool use.On GPT-4.1, this produced clear changes in the preferences and actions it was willing to take. In the most striking case, the…

Zac Boring 6 months ago Analysis Essential

If Everyone Reads It, Nobody Dies - Course Launch

via LessWrong AI [19] — tl;dr: Lens Academy offers a new course introducing ASI x-risk for AI safety newcomers, centered around the book IABIED. We share our hypothesis of why IABIED seems more appreciated by AI Safety newbies than by AI Safety insiders.Lens Academy's new intro…

Zac Boring 6 months ago Analysis

Does your AI perform badly because you — you, specifically — are a bad person

via LessWrong AI [4] — Claude really got me lately.I’d given it an elaborate prompt in an attempt to summon an AGI-level answer to my third-grade level question. Embarrassingly, it included the phrase, “this work might be reviewed by probability theorists, who are very…

Zac Boring 6 months ago Analysis Essential

AI #165: In Our Image

via Substack Zvi [999] — This was the week of Claude Opus 4.7.

Zac Boring 6 months ago Research

From Actions to Understanding: Conformal Interpretability of Temporal Concepts in LLM Agents

via ArXiv cs.AI [5] — Large Language Models (LLMs) are increasingly deployed as autonomous agents capable of reasoning, planning, and acting within interactive environments. Despite their growing capability to perform multi-step reasoning and decision-making tasks, internal…

Zac Boring 6 months ago Analysis Essential

Opus 4.7 Part 3: Model Welfare

via Substack Zvi [999] — It is thanks to Anthropic that we get to have this discussion in the first place.

Zac Boring 6 months ago Research

ARES: Adaptive Red-Teaming and End-to-End Repair of Policy-Reward System

via ArXiv cs.AI [5] — Reinforcement Learning from Human Feedback (RLHF) is central to aligning Large Language Models (LLMs), yet it introduces a critical vulnerability: an imperfect Reward Model (RM) can become a single point of failure when it fails to penalize unsafe…

Zac Boring 6 months ago Research Essential

A "Lay" Introduction to "On the Complexity of Neural Computation in Superposition"

via Alignment Forum [999] — This is a writeup based on a lightning talk I gave at an InkHaven hosted by Georgia Ray, where we were supposed to read a paper in about an hour, and then present what we learned to other participants.Introduction and BackgroundSo. I foolishly thought…

Zac Boring 6 months ago Analysis Essential

Opus 4.7 Part 2: Capabilities and Reactions

via Substack Zvi [999] — Claude Opus 4.7 raises a lot of key model welfare related concerns.

Zac Boring 6 months ago Industry

Inventor recalls eye imaging breakthrough

via MIT Technology Review [4] — If you’ve been to an eye doctor and had an image taken of the inside of your eye, chances are good it was done with optical coherence tomography (OCT)—a technology invented by clinician-scientist David Huang ’85, SM ’89, PhD ’93, and now used in…

Zac Boring 6 months ago Research Essential

$50 million a year for a 10% chance to ban ASI

via Alignment Forum [999] — ControlAI's mission is to avert the extinction risks posed by superintelligent AI. We believe that in order to do this, we must secure an international prohibition on its development. We're working to make this happen through what we believe is the…

Zac Boring 6 months ago Research

Governing the Agentic Enterprise: A Governance Maturity Model for Managing AI Agent Sprawl in Business Operations

via ArXiv cs.AI [4] — The rapid adoption of agentic AI in enterprise business operations--autonomous systems capable of planning, reasoning, and executing multi-step workflows--has created an urgent governance crisis. Organizations face uncontrolled agent sprawl: the…

Zac Boring 6 months ago Analysis Essential

Opus 4.7 Part 1: The Model Card

via Substack Zvi [999] — Less than a week after completing coverage of Claude Mythos, here we are again as Anthropic gives us Claude Opus 4.7.

Zac Boring 6 months ago Analysis

Resources for starting and growing an AI safety org

via LessWrong AI [5] — It seems that AI safety is at least partly bottlenecked by a lack of orgs. To help address that, we’ve added a page to AISafety.com aimed at lowering the friction for starting one: AISafety.com/founders.This page was built largely as the result of a…

Zac Boring 6 months ago Research

LLM Reasoning Is Latent, Not the Chain of Thought

via ArXiv cs.AI [5] — This position paper argues that large language model (LLM) reasoning should be studied as latent-state trajectory formation rather than as faithful surface chain-of-thought (CoT). This matters because claims about faithfulness, interpretability, reasoning…

Zac Boring 6 months ago Analysis

Reevaluating "AGI Ruin: A List of Lethalities" in 2026

via LessWrong AI [7] — It's been about four years since Eliezer Yudkowsky published AGI Ruin: A List of Lethalities, a 43-point list of reasons the default outcome from building AGI is everyone dying. A week later, Paul Christiano replied with Where I Agree and Disagree with…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →