pDoom (Page 13)

Latest reporting Updated hourly

Zac Boring 2 months ago Analysis

Policy career planning in the age of imminent superintelligence

via LessWrong AI [4] — Nearly all career advice rests on an unstated assumption that the world your career operates in will look roughly like the world you trained for. The idea was that if you spend six years studying in a PhD, the field you studied will still be there and…

Zac Boring 2 months ago Research Essential

AISN #79: OpenAI Agents’ Covert Cooperation Before Cyberattacks

via Center for AI Safety Newsletter [999] — Also, the White House’s decision not to release its AI framework publicly

Zac Boring 2 months ago Industry

The Download: how people really use AI, and Flock’s design choices

via MIT Technology Review [4] — This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. We still don’t know how people are really using AI AI companies like Anthropic and OpenAI regularly publish…

Zac Boring 2 months ago Industry

We still don’t know how people are really using AI

via MIT Technology Review [4] — AI companies like Anthropic and OpenAI regularly publish reports on how people are using products like Claude and ChatGPT, but they only release the data they want us to see, AI researchers say. “There is no independent source to corroborate it,”…

Zac Boring 2 months ago Analysis

For Claude, capability and dispreferring CDT are the ~same thing. Much more so than for GPT.

via LessWrong AI [3] — We've previously reported that decision-theoretic capabilities and favoring EDT/generalised-one-boxing over CDT correlate in LLMs (both measured by DTBench). (Note that EDT, for the most part, doesn't come apart from FDT / UDT on DTBench.[1]) Anthropic…

Zac Boring 2 months ago Research

Position: AI Governance Needs ISO-like Interoperability Protocols, Not Just Laws

via ArXiv cs.AI [7] — As Artificial Intelligence (AI) systems become deeply integrated into critical global infrastructure, the urgency for robust governance frameworks has intensified. However, current approaches, led by jurisdiction-specific laws, policies, and voluntary…

Zac Boring 2 months ago Research

Position: AI Lock-In Is in Progress, and We Must Be Prepared

via ArXiv cs.AI [5] — AI safety research has mainly focused on two areas: technical alignment (ensuring AI systems produce human-aligned outputs) and the regulation of generative AI's societal impacts (including unemployment risk and labor market disruption). However, an equally…

Zac Boring 2 months ago Research

Global AI Regulations for FAIR and Ethics in High-Risk Use Cases: A Comparative Review

via ArXiv cs.AI [6] — AI governance is shifting from voluntary ethics to enforceable, risk-based regulation, yet cross-jurisdictional divergence creates compliance uncertainty for operators of high-stakes AI. We present a comparative matrix for the EU, US, and China that maps…

Zac Boring 2 months ago Industry

Anthropic explains how Claude’s invisible text watermarks will work

via The Verge AI [4] — Anthropic has clarified how it's planning to apply invisible watermarks to Claude-generated text in order to comply with Europe's AI transparency rules. On Friday, Anthropic announced that Claude's text marking system is "a version of the SynthID-Text…

Zac Boring 2 months ago Research

Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors

via ArXiv cs.AI [5] — High-confidence errors in large language models are often treated as evidence of fragile internal inference. We study a different possibility: stable miscalibration, where a confident wrong answer remains locally stable under small perturbations. We combine…

Zac Boring 2 months ago Analysis

Three thoughts on civilisational handoff

via LessWrong AI [4] — What happens when humans put AIs in charge of civilisationally important decisions? A frontier AI company might hand over internal decisions (R&D, safety, deployment) or external decisions (government relations, public relations, philanthropy), or both. We…

Zac Boring 2 months ago Industry Essential

Rogue AI aren’t science fiction anymore

via The Verge AI [10] — This is The Stepback, a weekly newsletter breaking down one essential story from the tech world. For more on AI safety, follow Robert Hart. The Stepback arrives in our subscribers' inboxes at 8AM ET. Opt in for The Stepback here. How it started It all…

Zac Boring 2 months ago Research Essential

Does DiffusionGemma do latent reasoning?

via Alignment Forum [999] — TL;DR Google DeepMind's recent model DiffusionGemma (DG) generates text via diffusion, meaning many diffusion steps happen before generating the final output. In particular, these diffusion steps carry vectors in addition to tokens. If we cannot…

Zac Boring 2 months ago Analysis Essential

On Dwarkesh Patel's Podcast With Ryan Greenblatt

via Substack Zvi [999] — Some podcasts are self-recommending enough that I look to break them down if I have the chance.

Zac Boring 2 months ago Analysis

Rerunning AI safety papers on every frontier release would be pretty easy and valuable

via LessWrong AI [5] — tl;dr: Some important AI safety research is never rerun on the newest models. There are probably cases where this would be valuable and a single well-positioned researcher could likely do this with sufficient funding.This summer, Second Look Research (SLR)…

Zac Boring 2 months ago Research

Position: We Need Practical AI Alignment Methods to Mirror Human Reasoning

via ArXiv cs.AI [5] — AI systems are increasingly employed as decision aids, decision delegates, or autonomous decision-makers. This position paper argues that in many settings, particularly high-stakes decision-making, we need accurate cognitively-aligned AI systems that reason…

Zac Boring 2 months ago Research

Position: The Alignment Community is Unintentionally Building a Censor's Toolkit

via ArXiv cs.AI [5] — This position paper argues that modern AI alignment methods - originally designed to prevent harmful output - are dual-use technologies that may easily be misused by malicious actors for censorship and manipulation. By mapping current alignment techniques…

Zac Boring 2 months ago Research

Diagnostic Foundation for Evaluating LLMs' Research Integrity as Co-Scientists

via ArXiv cs.AI [4] — Language models are increasingly deployed as co-scientists, yet their ability to uphold research integrity under institutional pressure remains unmeasured. We introduce IntegrityBench, a benchmark evaluating misconduct classification, ethical action…

Zac Boring 2 months ago Industry

OpenAI and Anthropic in price war as Chinese AI rivals gain ground

via Ars Technica [4] — US groups release cheaper models after new challenges to their trillion-dollar ambitions.

Zac Boring 2 months ago Analysis

Automated alignment runs are hard to study!

via LessWrong AI [4] — TL;DR: This post presents three case studies of automated alignment research runs at Arcadia Impact. We use these case studies to emphasise the following takeaways:It is hard to parse auto-research runs! Each run produces a couple of hundred pull requests…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →