Research - pDoom (Page 13)

Zac Boring 7 months ago Research

Working Paper: Towards a Category-theoretic Comparative Framework for Artificial General Intelligence

via ArXiv cs.AI [8] — AGI has become the Holly Grail of AI with the promise of level intelligence and the major Tech companies around the world are investing unprecedented amounts of resources in its pursuit. Yet, there does not exist a single formal definition and only some…

Zac Boring 7 months ago Research

MediHive: A Decentralized Agent Collective for Medical Reasoning

via ArXiv cs.AI [6] — Large language models (LLMs) have revolutionized medical reasoning tasks, yet single-agent systems often falter on complex, interdisciplinary problems requiring robust handling of uncertainty and conflicting evidence. Multi-agent systems (MAS) leveraging…

Zac Boring 7 months ago Research

The AI Doc: Your Questions Answered

via MIRI [999] — So you’ve just seen The AI Doc, and you suddenly have questions, lots of them. The 104-minute documentary (currently in theaters) takes viewers on a fast-paced tour through the many dimensions of the AI problem, featuring interviews from a wide range of experts.…

Zac Boring 7 months ago Research

Trust as Monitoring: Evolutionary Dynamics of User Trust and AI Developer Behaviour

via ArXiv cs.AI [8] — AI safety is an increasingly urgent concern as the capabilities and adoption of AI systems grow. Existing evolutionary models of AI governance have primarily examined incentives for safe development and effective regulation, typically representing users'…

Zac Boring 7 months ago Research

ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence

via ArXiv cs.AI [4] — We introduce ARC-AGI-3, an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments in which agents must explore, infer goals, build internal models of environment dynamics, and plan effective action sequences…

Zac Boring 7 months ago Research

Test your best methods on our hard CoT interp tasks

via Alignment Forum [999] — Authors: Daria Ivanova, Riya Tyagi, Arthur Conmy, Neel NandaDaria and Riya are co-first authors. This work was done during Neel Nanda’s MATS 9.0. Claude helped write code and suggest edits for this post.TL;DR One of our best safety techniques right…

Zac Boring 7 months ago Research

A Toy Environment For Exploring Reasoning About Reward

via Alignment Forum [999] — tldr: We share a toy environment that we found useful for understanding how reasoning changed over the course of capabilities-focused RL. Over the course of capabilities-focused RL, the model biases more strongly towards reward hints over direct…

Zac Boring 7 months ago Research

Intelligence Inertia: Physical Principles and Applications

via ArXiv cs.AI [3] — While Landauer's principle establishes the fundamental thermodynamic floor for information erasure and Fisher Information provides a metric for local curvature in parameter space, these classical frameworks function effectively only as approximations within…

Zac Boring 7 months ago Research

Leveraging Natural Language Processing and Machine Learning for Evidence-Based Food Security Policy Decision-Making in Data-Scarce Making

via ArXiv cs.AI [4] — Food security policy formulation in data-scarce regions remains a critical challenge due to limited structured datasets, fragmented textual reports, and demographic bias in decision-making systems. This study proposes ZeroHungerAI, an integrated Natural…

Zac Boring 7 months ago Research

ProMAS: Proactive Error Forecasting for Multi-Agent Systems Using Markov Transition Dynamics

via ArXiv cs.AI [5] — The integration of Large Language Models into Multi-Agent Systems (MAS) has enabled the so-lution of complex, long-horizon tasks through collaborative reasoning. However, this collec-tive intelligence is inherently fragile, as a single logical fallacy can…

Zac Boring 7 months ago Research

MIRI Newsletter #125

via MIRI [999] — The AI Doc: Buy tickets and spread the word! On Thursday, March 26th, a major new AI documentary is coming out: The AI Doc: Or How I Became an Apocaloptimist. Tickets are on sale now. The movie is excellent, and we generally believe it belongs in the same tier…

Zac Boring 7 months ago Research

InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning

via ArXiv cs.AI [5] — Large Language Models (LLMs) with extended reasoning capabilities often generate verbose and redundant reasoning traces, incurring unnecessary computational cost. While existing reinforcement learning approaches address this by optimizing final response…

Zac Boring 7 months ago Research

Graph-Native Cognitive Memory for AI Agents: Formal Belief Revision Semantics for Versioned Memory Architectures

via ArXiv cs.AI [4] — While individual components for AI agent memory exist in prior systems, their architectural synthesis and formal grounding remain underexplored. We present Kumiho, a graph-native cognitive memory architecture grounded in formal belief revision semantics.…

Zac Boring 7 months ago Research

Metagaming matters for training, evaluation, and oversight

via Alignment Forum [999] — Following up on our previous work on verbalized eval awareness:we are sharing a post investigating the emergence of metagaming reasoning in a frontier training run.Metagaming is a more general, and in our experience a more useful concept, than…

Zac Boring 7 months ago Research

Mechanisms to Verify International Agreements about AI Development

via MIRI [999] — If world leaders agree to halt or limit AI development, they will need to verify that other nations are keeping their commitments. To this end, it helps to know where AI chips are, how they’re used, and what the AIs trained on them can do. In this post, we…

Zac Boring 7 months ago Research

“Act-based approval-directed agents”, for IDA skeptics

via Alignment Forum [999] — Summary / tl;drIn the 2010s, Paul Christiano built an extensive body of work on AI alignment—see the “Iterated Amplification” series for a curated overview as of 2018.One foundation of this program was an intuition that it should be possible to build…

Zac Boring 7 months ago Research

The Comprehension-Gated Agent Economy: A Robustness-First Architecture for AI Economic Agency

via ArXiv cs.AI [4] — AI agents are increasingly granted economic agency (executing trades, managing budgets, negotiating contracts, and spawning sub-agents), yet current frameworks gate this agency on capability benchmarks that are empirically uncorrelated with operational…

Zac Boring 7 months ago Research

Neural-Symbolic Logic Query Answering in Non-Euclidean Space

via ArXiv cs.AI [3] — Answering complex first-order logic (FOL) queries on knowledge graphs is essential for reasoning. Symbolic methods offer interpretability but struggle with incomplete graphs, while neural approaches generalize better but lack transparency. Neural-symbolic…

Zac Boring 7 months ago Research

New RFP on Interpretability from Schmidt Sciences

via Alignment Forum [999] — Request for ProposalsDeadline: Tuesday, May 26, 2026Schmidt Sciences invites proposals for a pilot program in AI interpretability. We seek new methods for detecting and mitigating deceptive behaviors from AI models, such as when models knowingly give…

Zac Boring 7 months ago Research

Measuring progress toward AGI: A cognitive framework

via DeepMind Blog [4] — We’re introducing a framework to measure progress toward AGI, and launching a Kaggle hackathon to build the relevant evaluations.