pDoom (Page 32)

Latest reporting Updated hourly

Zac Boring 5 months ago Analysis

Announcing Geodesic Research

via LessWrong AI [6] — We're a Cambridge, UK-based AI safety organisation that’s asking: how can we build the most robust alignment initialisations for capable LLMs?We’re one of the few non-profit organisations positioned to answer this question empirically. We have the…

Zac Boring 5 months ago Research Essential

Eval Cooperativeness May Be a Scalable Mitigation for Eval Gaming

via Alignment Forum [999] — Behavioral evaluations may become worthless, which we think would be a disaster. Smart misaligned models may realize they are being evaluated ("eval awareness") and then act to look good to us so we don't realize they're misaligned ("eval gaming"). We…

Zac Boring 5 months ago Research Essential

Full automation of AI R&D probably yields a large speed up even without a software-only singularity

via Alignment Forum [999] — This is a somewhat technical note. By "software-only singularity", I mean that, after full automation of AI R&D, progress gets faster and faster due to smarter AIs driving increasingly fast rates of improvement in algorithms (overcoming diminishing…

Zac Boring 5 months ago Analysis

Quantitative AI risk assessment: a starting point

via LessWrong AI [4] — Current AI risk management relies on qualitative approaches, much like nuclear safety before 1975. We propose a shift to quantitative risk modeling, following the approach that transformed nuclear safety. We propose a methodology and demonstrate it by…

Zac Boring 5 months ago Industry

AI tried to bury this politician — now people have actually heard of him

via The Verge AI [4] — By the time that the Democratic primary for New York's 12th congressional district wraps up in June, Anthropic and OpenAI will have spent millions on their battle over the political future of AI: who gets to regulate it, or who will be punished for trying…

Zac Boring 5 months ago Industry

The Pope isn’t AGI-pilled

via The Verge AI [6] — On Monday, Pope Leo XIV unveiled an encyclical letter addressing the societal implications of artificial intelligence. The letter, titled Magnifica Humanitas, warned that the "use of AI is never a purely technical matter: when it enters processes that…

Zac Boring 5 months ago Research

Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems

via ArXiv cs.AI [4] — Long-lived AI agents are increasingly deployed as persistent operational systems, yet they are still evaluated like freshly initialized models. Day-one benchmarks miss a basic systems question: how long does an agent remain reliable after deployment? Even…

Zac Boring 5 months ago Analysis Essential

RTMH: Pope Leo's Magnifica Humanitas on AI

via Substack Zvi [999] — His holiness has spoken, frequently about AI.

Zac Boring 5 months ago Analysis

Cognitive Security as an AI Safety Cause Area

via LessWrong AI [5] — As AI systems become more capable, the cognitive security of humans will be increasingly at risk. By cognitive security, I mean the ability of humans to maintain control over their beliefs and actions.Cognitive security could be compromised in several…

Zac Boring 5 months ago Analysis Essential

Linkpost: New Vatican Encyclical on AI Governance

via LessWrong AI [9] — Pope Leo XIV has released a new, 42k-word encyclical laying out the Vatican's position on many AI safety topics. You can read the full thing here, or read the Vatican's press release here, or coverage in the NY Times, or perhaps consider having an LLM read…

Zac Boring 5 months ago Analysis

We made a map of the doom debate

via LessWrong AI [5] — This was produced as a part of the AI Safety Camp 2026 "Assumptions of the Doom Debate" project, led by Sean Herrington, who was also the lead author on this post. The other participants have equal contributions and are listed in no particular order. It is…

Zac Boring 5 months ago Analysis

Will we really put data centers in space?

via LessWrong AI [3] — AbstractSeveral major technology companies have announced plans to operate AI data centers in orbit. Elon Musk recently claimed: “the lowest-cost place to put AI will be space […] within two years, maybe three.” If a meaningful fraction of new AI compute…

Zac Boring 5 months ago Analysis

PLA Daily Translation: Reflections on Warfare Brought by AGI

via LessWrong AI [4] — Source“Reflections on Warfare Brought by AGI” (AGI带来的战争思考)Source: PLA Daily (解放军报)Date: January 21, 2025Authors: Rong Ming (荣明), Hu Xiaofeng (胡晓峰)IntroductionPlease feel free to skip to the translation, about halfway down, though I would recommend reading…

Zac Boring 5 months ago Analysis

Out-of-Context Reasoning (OOCR) in LLMs: A Short Primer and Reading List

via LessWrong AI [6] — Out-of-context reasoning (OOCR) is a concept relevant to LLM generalization and AI alignment. Also available as a PDF. Contents What is OOCR? Examples Papers Videos What is out-of-context reasoning for LLMs? It's when an LLM reaches a conclusion that…

Zac Boring 5 months ago Analysis

Learned Chain-of-Thought Obfuscation Generalises to Unseen Tasks

via LessWrong AI [3] — TL;DRTraining against a CoT or summary-only monitor can lead to obfuscation of dangerous reasoning in unseen tasks. This strengthens the “don’t train against a monitor” claims.Figure 1. A Two prior results: penalising the CoT or final response produces…

Zac Boring 5 months ago Analysis Essential

Gemini 3.5 Flash Looks Good For How Fast It Is

via Substack Zvi [999] — Google once again has a model worth at least some consideration.

Zac Boring 5 months ago Research Essential

The Erdős Proof and AI Capabilities

via MIRI [999] — View the official memo here. An internal model at OpenAI has autonomously disproved a central conjecture in discrete geometry, a mathematical field with applications in cryptography, wireless device communication, and medical imaging. The proof relates to a…

Zac Boring 5 months ago Industry

Why would you disrespect your favorite artist with an AI remix?

via The Verge AI [4] — AI covers and remixes of songs are already a blight on the internet. Spotify, YouTube, TikTok, and Instagram are awash in flat reggae versions of "Smells Like Teen Spirit," dinky country renditions of The Weeknd, and monotonous Motown reimaginings of…

Zac Boring 5 months ago Industry

OpenAI named a Leader in enterprise coding agents by Gartner

via OpenAI Blog [6] — OpenAI is named a leader in the 2026 Gartner Magic Quadrant for Enterprise AI Coding Agents, with Codex recognized for innovation and enterprise-scale deployment.

Zac Boring 5 months ago Analysis

Do AI Risks Require Extraordinary Government Intervention?

via AI Snake Oil [7] — Let’s not skip the hard work of AI governance

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →