pDoom (Page 35)

Latest reporting Updated hourly

Zac Boring 5 months ago Research

Understanding Annotator Safety Policy with Interpretability

via ArXiv cs.AI [3] — Safety policies define what constitutes safe and unsafe AI outputs, guiding data annotation and model development. However, annotation disagreement is pervasive and can stem from multiple sources such as operational failures (annotators misunderstand or…

Zac Boring 5 months ago Analysis

Is ProgramBench Impossible?

via LessWrong AI [3] — ProgramBench is a new coding benchmark that all frontier models spectacularly fail. We’ve been on a quest for “hard benchmarks” for a while so it’s refreshing to see a benchmark where top models do badly. Unfortunately, ProgramBench has one big problem:…

Zac Boring 5 months ago Analysis Essential

Claude Code, Codex and Agentic Coding #8

via Substack Zvi [999] — When I started this series, everyone was going crazy for coding agents.

Zac Boring 5 months ago Analysis Essential

The AI industry is where banking was in 2006. (We're hiring)

via LessWrong AI [8] — TL;DR; CeSIA, the French Center for AI Safety is recruiting. French not necessary. Apply by 22 May 2026; Paris or remote in Europe/UK.On August 27, 2005, at an annual symposium in Jackson Hole, Raghuram Rajan, then chief economist of the International…

Zac Boring 5 months ago Research Essential

Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations

via Alignment Forum [999] — AbstractWe introduce Natural Language Autoencoders (NLAs), an unsupervised method for generating natural language explanations of LLM activations. An NLA consists of two LLM modules: an activation verbalizer (AV) that maps an activation to a text…

Zac Boring 5 months ago Research Essential

Mechanistic estimation for wide random MLPs

via Alignment Forum [999] — This post covers joint work with Wilson Wu, George Robinson, Mike Winer, Victor Lecomte and Paul Christiano. Thanks to Geoffrey Irving and Jess Riedel for comments on the post. In ARC's latest paper, we study the following problem: given a randomly…

Zac Boring 5 months ago Analysis Essential

AI #167: The Prior Restraint Era Begins

via Substack Zvi [999] — The era of training frontier models and then releasing them whenever you wanted?

Zac Boring 5 months ago Analysis

Many individual CEVs are probably quite bad

via LessWrong AI [4] — I was thinking about Habryka's article on Putin's CEV, but I am posting my response here, because the original article is already 3 weeks old.I am not sure how exactly a person's CEV is defined. "If we knew everything and could self-modify" seems…

Zac Boring 5 months ago Analysis

x-risk-themed

via LessWrong AI [5] — Sometimes, a friend who works around here, at an x-risk-themed organisation, will think about leaving their job. They’ll ask a group of people “what should I do instead?”. And everyone will chime in with ideas for other x-risk-themed orgs that they could…

Zac Boring 5 months ago Analysis

What if LLMs are mostly crystallized intelligence?

via LessWrong AI [5] — SummaryLLMs are better at developing crystallized intelligence than fluid intelligence. That is: LLM training is good at building crystallized intelligence by learning patterns from training data, and this is sufficient to make them surprisingly skillful…

Zac Boring 5 months ago Analysis Essential

What is Anthropic?

via Substack Zvi [999] — What is Anthropic?

Zac Boring 5 months ago Research

Terminus-4B: Can a Smaller Model Replace Frontier LLMs at Agentic Execution Tasks?

via ArXiv cs.AI [4] — Modern coding agents increasingly delegate specialized subtasks to subagents, which are smaller, focused agentic loops that handle narrow responsibilities like search, debugging or terminal execution. This architectural pattern keeps the main agent's…

Zac Boring 5 months ago Analysis Essential

The AI Ad-Hoc Prior Restraint Era Begins

via Substack Zvi [999] — The White House has ordered Anthropic not to expand access to Mythos, and is at least seriously considering a complete about-face of American Frontier AI policy into a full prior restraint regime, where anyone wishing to release a highly capable new…

Zac Boring 5 months ago Research Essential

[Linkpost] Interpreting Language Model Parameters

via Alignment Forum [999] — This is the latest work in our Parameter Decomposition agenda. We introduce a new parameter decomposition method, adVersarial Parameter Decomposition (VPD)[1] and decompose the parameters of a small[2] language model with it. VPD greatly improves on…

Zac Boring 5 months ago Research Essential

Motivated reasoning, confirmation bias, and AI risk theory

via Alignment Forum [999] — Of the fifty-odd biases discovered by Kahneman, Tversky, and their successors, forty-nine are cute quirks, and one is destroying civilization. This last one is confirmation bias.- From Scott Alexander's review of Julia Galef's The Scout Mindset.…

Zac Boring 5 months ago Industry

Google’s AI architect lived rent-free in Elon Musk’s head

via The Verge AI [4] — About a week into the Musk v. Altman trial, we've heard from some of the most powerful people in tech - including OpenAI president Greg Brockman, Elon Musk's fixer Jared Birchall, and Musk himself. But one of the most prominent characters is hovering…

Zac Boring 5 months ago Research

Understanding Emergent Misalignment via Feature Superposition Geometry

via ArXiv cs.AI [6] — Emergent misalignment, where fine-tuning on narrow, non-harmful tasks induces harmful behaviors, poses a key challenge for AI safety in LLMs. Despite growing empirical evidence, its underlying mechanism remains unclear. To uncover the reason behind this…

Zac Boring 5 months ago Industry

OpenAI and PwC collaborate to reimagine the office of the CFO

via OpenAI Blog [6] — OpenAI and PwC are partnering to help enterprises use AI agents to automate finance workflows, improve forecasting, strengthen controls, and modernize the CFO function.

Zac Boring 5 months ago Analysis Essential

Housing Roundup #15: The War Against Renters

via Substack Zvi [999] — So many are under the strange belief that there is something terrible about not owning the house in which you live.

Zac Boring 5 months ago Industry

The creator of Roomba is back with a furry robot companion

via The Verge AI [4] — Colin Angle, the maker of the Roomba and the man who helped put 50 million household robots into people's homes, is back with a new robot. But this one is designed as a companion, not a cleaner. The first robot from Angle's new company, Familiar Machines &…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →