pDoom (Page 17)

Latest reporting Updated hourly

Zac Boring 3 months ago Research Essential

Thousand-dimensional structure

via Alignment Forum [999] — Summary: One area we plan to explore at Resolution is personas and character training, operationalized as finding and controlling low-dimensional structure in models that emerges in pretraining and flows through post-training to superintelligence. The…

Zac Boring 3 months ago Analysis Essential

AI #179 Part 1: A Louder Fire Alarm for General Intelligence

via Substack Zvi [999] — What a week.

Zac Boring 3 months ago Research

Position: Evaluation Scores Are Perishable Knowledge Claims

via ArXiv cs.AI [5] — Evaluation methodologies for language models increasingly combine multiple signals, from automated metrics and LLM-as-judge ratings to human assessments and benchmark suite results. When these signals are aggregated via averaging, evaluation confidence can…

Zac Boring 3 months ago Research Essential

Imprecise beliefs: a tiny introduction

via Alignment Forum [999] — Richard Ngo challenged me to set a time box and write down as many of the most important features of my formal epistemology as I can in one sitting. Here goes.Where probability distributions fail......to express beliefsThere is no probability…

Zac Boring 3 months ago Industry

xAI’s last-minute scramble to stop Minnesota’s anti-nudification app law

via The Verge AI [4] — xAI is suing Minnesota Attorney General Keith Ellison over a law passed back in May that broadly targets "nudification" apps, claiming that the statute's punitive provisions leave the company with "no practical choice but to restrict Grok Imagine's…

Zac Boring 3 months ago Research Essential

Value Generalisation 3: Pre-aligned AIs

via Alignment Forum [999] — When we get explicit strong generalisation to work (see the first post on the matter and the second) my dream would be to create pre-aligned generalising AIs.Think about the usual conflict between alignment and capabilities, between doing the right…

Zac Boring 3 months ago Research Essential

Value Generalisation 2: The Missing Hole in AIs’ abilities

via Alignment Forum [999] — A human superpower hidden from even ourselvesI though GPT 3.5 was on the verge of Artificial General Intelligence (AGI). It certainly seemed that way – it could combine and extend ideas in ways that were far beyond narrow rigid computing. Sure, it had…

Zac Boring 3 months ago Research Essential

Value Generalisation 1: a Research and Deployment Program

via Alignment Forum [999] — I’m looking for people, advice, critiques, and funding to build a research program on value generalisation – the ability of an AI to correctly extend human values and preferences to situations neither it nor we have seen before. My ongoing research…

Zac Boring 3 months ago Analysis Essential

Frontier Lab Employee Open Letter Calls For Being Able to Pace the Frontier

via Substack Zvi [999] — The most important open letter in years dropped yesterday.

Zac Boring 3 months ago Industry

How enabling two settings tripled our scores on the ARC-AGI-3 benchmark

via OpenAI Blog [4] — How two API settings improved GPT-5.6 performance on ARC-AGI-3, boosting scores and efficiency by retaining reasoning and enabling compaction.

Zac Boring 3 months ago Industry

We’re running out of reasons to ignore AI safety

via The Verge AI [7] — Earlier this month, OpenAI gave several of its AI models a task: complete a test designed to measure their cybersecurity capabilities. It put the systems in a sandboxed environment without an internet connection and set them off to work. What happened next…

Zac Boring 3 months ago Research Essential

LLM Scheming Inversely Scales with Pretraining Language Coverage

via ArXiv cs.AI [9] — With the growing capabilities of frontier models, AI alignment becomes increasingly critical in high-risk deployment settings. While recent work has empirically demonstrated in-context scheming -- the covert pursuit of misaligned objectives while feigning…

Zac Boring 3 months ago Analysis

Foundation Models for Oversight

via LessWrong AI [3] — To oversee an AI model, we'd ideally like to ask questions such as:What are important situations where the model sandbags?Does the model have an objective it wouldn't admit to if asked directly?Does the model treat a user differently once it infers…

Zac Boring 3 months ago Industry

AI leaders sign statement asking the government to do something about automated AI

via The Verge AI [4] — Employees of OpenAI and Anthropic, as well as Google, Meta, Thinking Machines, Microsoft, Mistral, and other leading AI labs, have written a statement to the US government supporting a potential slowdown of sorts for frontier AI development - or at least a…

Zac Boring 3 months ago Analysis Essential

Claude Opus 5 Is Highly Capable, But Is No Mythos

via Substack Zvi [999] — Claude Opus 5 is a weirder than usual release to evaluate, for two reasons.

Zac Boring 3 months ago Research Essential

Research directions in condensation: varieties of objectivity

via Alignment Forum [999] — This is the first part of a survey of various ways that I’d like to see work on the theory of condensation develop. Condensation is a mathematical theory dealing with the organization of descriptions of the world into conceptual parts; some of the…

Zac Boring 3 months ago Research

SF-AMS: Strategic Forgetting for Structured Memory in LLM Agent

via ArXiv cs.AI [5] — Managing long-context dependencies remains a primary bottleneck in LLM agents, as redundant and irrelevant information can degrade multi-step reasoning. Strategic Forgetting for Agent Memory Systems (SF-AMS) is proposed as a framework for maintaining…

Zac Boring 3 months ago Research

Concept-based Visual Counterfactual Explanations with Diffusion Models

via ArXiv cs.AI [4] — Visual counterfactual explanations aim to answer "what minimal change to this image would flip the model's prediction?", and are increasingly important as vision models are deployed in safety-critical domains (e.g., medicine). Existing diffusion-based…

Zac Boring 3 months ago Analysis Essential

Claude Opus 5: Model Welfare

via Substack Zvi [999] — If you are familiar with my previous posts on model welfare for new Claude models, you can skip the Introduction and The Story So Far.

Zac Boring 3 months ago Research Essential

RL & search is a terrifying way to build AGI (an FAQ)

via Alignment Forum [999] — Q1: What are you saying?A: My claim here is that if you build artificial general intelligence (AGI) via any algorithm that’s choosing actions via reinforcement learning (RL) and/or model-based search and planning—a giant chunk of your AI textbook—then…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →