pDoom (Page 34)

Latest reporting Updated hourly

Zac Boring 5 months ago Analysis

Convergent Abstraction Hypothesis

via LessWrong AI [4] — Tl;drConvergent abstraction hypothesis posits abstractions are often convergent in the sense of convergent evolution: different cognitive systems converge on the same abstraction, when facing similar selection pressures and learning in similar…

Zac Boring 5 months ago Industry

OpenAI’s Codex is now in the ChatGPT mobile app

via The Verge AI [4] — OpenAI is going to let users access Codex, its desktop AI tool that can write code and use apps on your computer, from the ChatGPT app on your phone. Following the surge in popularity for Anthropic's Claude Code, OpenAI has been working quickly to try and…

Zac Boring 5 months ago Research Essential

The safe-to-dangerous shift is a fundamental problem for eval realism; but also for measuring awareness

via Alignment Forum [999] — 1) The safe-to-dangerous shift is a fundamental problem for eval realismSuppose we have a capable and potentially scheming model, and before we deploy it, we want some evidence that it won’t do anything catastrophically dangerous once we deploy it. A…

Zac Boring 5 months ago Analysis Essential

AI #168: Not Leading the Future

via Substack Zvi [999] — This is what a lull looks like at this point.

Zac Boring 5 months ago Research

Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack

via ArXiv cs.AI [6] — Agent benchmarks have become the de facto measure of frontier AI competence, guiding model selection, investment, and deployment. However, reward hacking, where agents maximize a score without performing the intended task, emerges spontaneously in frontier…

Zac Boring 5 months ago Analysis

Most "inner work" looks like entertainment.

via LessWrong AI [4] — Imagine you’re looking for a personal trainer. You open one trainer’s webpage and read their testimonials: “I had an experience tied for the most intense experiences of my life”; “They do it all with fun, care, and a sense of humour.” You notice that none…

Zac Boring 5 months ago Analysis Essential

Cyber Lack of Security and AI Governance

via Substack Zvi [999] — The real recent story of AI has been the background work being done on Cybersecurity, as we process the Mythos Moment along with GPT-5.5, and figure out both how to patch the internet and what our new regulatory regime is going to look like.

Zac Boring 5 months ago Analysis Essential

Voters are surprisingly open to talking about AI risk

via LessWrong AI [14] — TL;DR: Voters are now surprisingly open to talking about existential risk from AI. This seems to have changed in the last 6 months. When campaigning for AI safety-friendly politicians (e.g., Alex Bores), we should talk more about AI in general, and about…

Zac Boring 5 months ago Research

RankQ: Offline-to-Online Reinforcement Learning via Self-Supervised Action Ranking

via ArXiv cs.AI [4] — Offline-to-online reinforcement learning (RL) improves sample efficiency by leveraging pre-collected datasets prior to online interaction. A key challenge, however, is learning an accurate critic in large state--action spaces with limited dataset coverage.…

Zac Boring 5 months ago Research Essential

Summary: An International Agreement to Prevent the Premature Creation of Artificial Superintelligence

via MIRI [999] — If anyone, anywhere builds a superhuman artificial intelligence using present methods, the most likely outcome is catastrophe. There have accordingly been widespread calls for an international agreement prohibiting the development of superintelligence. In…

Zac Boring 5 months ago Analysis Essential

Childhood and Education #18: Do The Math

via Substack Zvi [999] — We did reading yesterday.

Zac Boring 5 months ago Industry

Sam Altman says Elon Musk’s mind games were damaging OpenAI

via The Verge AI [6] — OpenAI CEO Sam Altman says Elon Musk did "huge damage" to the culture of the AI startup. During testimony as part of Musk's lawsuit against OpenAI, Altman said Musk required OpenAI president Greg Brockman and former chief scientist Ilya Sutskever to rank…

Zac Boring 5 months ago Research

Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteria

via ArXiv cs.AI [6] — Aligning multimodal generative models with human preferences demands reward signals that respect the compositional, multi-dimensional structure of human judgment. Prevailing RLHF approaches reduce this structure to scalar or pairwise labels, collapsing…

Zac Boring 5 months ago Analysis

The Iliad Intensive Course Materials

via LessWrong AI [5] — We are releasing the course materials of the Iliad Intensive, a new month-long and full-time AI Alignment course that runs in-person every second month. The course targets students with strong backgrounds in mathematics, physics, or theoretical computer…

Zac Boring 5 months ago Analysis Essential

Childhood And Education #17: Is Our Children Reading

via Substack Zvi [999] — Reading is the most fundamental thing in education.

Zac Boring 5 months ago Research Essential

Empowerment, corrigibility, etc. are simple abstractions (of a messed-up ontology)

via Alignment Forum [999] — 1.1 Tl;drAlignment is often conceptualized as AIs helping humans achieve their goals: AIs that increase people’s agency and empowerment; AIs that are helpful, corrigible, and/or obedient; AIs that avoid manipulating people. But that last…

Zac Boring 5 months ago Research

Hidden Coalitions in Multi-Agent AI: A Spectral Diagnostic from Internal Representations

via ArXiv cs.AI [5] — Collections of interacting AI agents can form coalitions, creating emergent group-level organization that is critical for AI safety and alignment. However, observing agent behavior alone is often insufficient to distinguish genuine informational coupling…

Zac Boring 5 months ago Research Essential

Clarifying the role of the behavioral selection model

via Alignment Forum [999] — This is a brief elaboration on The behavioral selection model for predicting AI motivations, based on some feedback and thoughts I’ve had since publishing. Written quickly in a personal capacity.The main focus of this post is clarifying the basic…

Zac Boring 5 months ago Analysis

Why You Can't Use Your Right to Try

via LessWrong AI [4] — The Availability Problem:Imagine you have cancer, or chronic pain, or a progressive degenerative disease of some sort. You have exhausted the traditional treatment options available to you, and none of them have worked. However, there are treatments that…

Zac Boring 5 months ago Research

Intelligent CCTV for Urban Design: AI-Based Analysis of Soft Infrastructure at Intersections

via ArXiv cs.AI [4] — Artificial intelligence (AI) and computer vision are transforming transportation data collection. This study introduces an AI-enabled analytics framework leveraging existing CCTV infrastructure to evaluate the impact of soft interventions, such as temporary…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →