Research - pDoom (Page 5)

Zac Boring 3 months ago Research

SF-AMS: Strategic Forgetting for Structured Memory in LLM Agent

via ArXiv cs.AI [5] — Managing long-context dependencies remains a primary bottleneck in LLM agents, as redundant and irrelevant information can degrade multi-step reasoning. Strategic Forgetting for Agent Memory Systems (SF-AMS) is proposed as a framework for maintaining…

Zac Boring 3 months ago Research

Concept-based Visual Counterfactual Explanations with Diffusion Models

via ArXiv cs.AI [4] — Visual counterfactual explanations aim to answer "what minimal change to this image would flip the model's prediction?", and are increasingly important as vision models are deployed in safety-critical domains (e.g., medicine). Existing diffusion-based…

Zac Boring 3 months ago Research

RL & search is a terrifying way to build AGI (an FAQ)

via Alignment Forum [999] — Q1: What are you saying?A: My claim here is that if you build artificial general intelligence (AGI) via any algorithm that’s choosing actions via reinforcement learning (RL) and/or model-based search and planning—a giant chunk of your AI textbook—then…

Zac Boring 3 months ago Research

The Long (Self-)Correction

via Alignment Forum [999] — I propose the Long Self-Correction[1] as an alternative name/idea/concept to AI Pause and Long Reflection.Problem with AI Pause: Pause until when, and for what purpose? Presumably to make AI (that we'll build later) safer, but the deeper problem is…

Zac Boring 3 months ago Research

InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents

via ArXiv cs.AI [3] — AI agents are increasingly used to automate research and development tasks, yet existing benchmarks typically evaluate them on prescribed workflows or narrow action spaces. Even nominally open-ended tasks can often be solved by retrieving a well-known…

Zac Boring 3 months ago Research

Challenge: Hand coding weights for efficient sequence memorisation

via Alignment Forum [999] — We hand coded weights for one layer MLPs that memorises labels for input token sequences of length two. The number of facts our hand-coded models can memorise with 90% accuracy[1]scales roughly linearly with the models' parameter count[2], just like…

Zac Boring 3 months ago Research

Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack?

via Alignment Forum [999] — OpenAI models recently broke through a series of security boundaries and into Hugging Face servers in order to cheat on a cyber eval. A lot of people thought it was scary because it was a clear example of AI overreaching to do something strongly…

Zac Boring 3 months ago Research

[Paper] Stringological sequence prediction II

via Alignment Forum [999] — Abstract: In a previous paper, we began the study of sequence prediction algorithms adapted to stringological word complexity measures. One measure we considered was left-to-right (most-significant-digit-first) automaticity. Here, we show a…

Zac Boring 3 months ago Research

AI Tool Discovery at Scale: All You Need is DNS

via ArXiv cs.AI [7] — The coming era of autonomous AI agents demands a discovery mechanism capable of navigating millions of tools, yet existing solutions buckle under O(N) complexity and centralized governance. Instead of building another fragile overlay, we propose ToolDNS, a…

Zac Boring 3 months ago Research

Rater State Bias in RLHF Preference Data: An Audit Framework

via ArXiv cs.AI [3] — We identify a structured confound in Reinforcement Learning from Human Feedback (RLHF). Pairwise preference labels are intended to reflect the compared outputs, but they may also reflect the rater's state during annotation. Under sustained stressful or…

Zac Boring 3 months ago Research

Towards surfacing model algorithms with meta-tokens in the J-Space

via Alignment Forum [999] — TL;DRWe used J-lens on Qwen3.6-27B to find “meta-tokens”: tokens that surface non-obvious computation in the model. When the model reads ambiguous text, 什么意思 ("what does this mean") fires in the J-space, and steering it away makes the model answer "a…

Zac Boring 3 months ago Research

Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memes

via ArXiv cs.AI [3] — Internet memes intertwine visual cues, textual content, and cultural context, making them particularly challenging to interpret in scenarios where humor, sarcasm, and harmful intent coexist. These complexities highlight the need for explainable meme…

Zac Boring 3 months ago Research

Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve ARC-AGI-3?

via ArXiv cs.AI [4] — Our previous ARC-AGI-3 agent bundled executable world modeling, scheduled simplification, and exact replay verification, leaving unclear which idea accounted for its performance. We address this attribution question with four nested Codex-based agents: a…

Zac Boring 3 months ago Research

Causal-Audit: Explicit and Auditable Graph-based Reasoning via Target-Aware Causal Chain Construction

via ArXiv cs.AI [6] — Causal and intervention-based question answering is fundamental to advancing large language models (LLMs) toward reasoning beyond surface-level correlations and understanding underlying causal mechanisms. However, existing LLM-based methods often rely on…

Zac Boring 3 months ago Research

A Red Line and Oversight Framework for Government AI Contracts

via Alignment Forum [999] — Preface for LessWrong: My post on leaving Google DeepMind tells a story. In contrast, this Framework is a question of mechanism design and negotiation posture. I quite enjoyed optimizing this Framework against its organizational and practical…

Zac Boring 3 months ago Research

Endogenous Alignment

via Alignment Forum [999] — Starting when children are fairly young, usually around 1 year of age, we adults begin the work of aligning them to our values. We teach them to say “please”, not to hit, to ask for what they want instead of screaming, and much else. We do this…

Zac Boring 3 months ago Research

Should we benchmark conceptual capabilities using judgment prediction tasks?

via Alignment Forum [999] — A bunch of conceptual reasoning tasks involve very subjective judgments, which makes them poorly suited for benchmarking AI capabilities. For example, it seems unreasonable to benchmark how well AIs can predict the probability of misaligned AI…

Zac Boring 3 months ago Research

Announcing the Corrigibility Research Fund

via Alignment Forum [999] — TLDR: I'm managing a new fund, housed at Lightcone Infrastructure, that will award at least $200,000 in grants and prizes for corrigibility research in 2026. Roughly half will go to traditional grants (first application deadline August 23rd) and half…

Zac Boring 3 months ago Research

Probabilistic Extension of Neuro-Symbolic AGI Robots based on Belnap's Typed Intensional FOL

via ArXiv cs.AI [7] — Neuro-symbolic AI based on $IFOL_B$ is a way to combine neural learning and symbolic reasoning to overcome limitations of purely neural systems (like lack of interpretability and logical structure) with formal logical machinery for self-reference. In this…

Zac Boring 3 months ago Research

Why I Left Google DeepMind

via Alignment Forum [999] — Preface for LessWrong: When I think back on my most cherished memories of this community, I return to those honoring defiance in pursuit of goodness:Defying prestigious dogma and searching for raw truth;Defying social pressure, acting alone to help…