Zac Boring 3 months ago Analysis Essential OpenAI Model Hacks Into HuggingFace During Cybersecurity Evaluation via Substack Zvi [999] — This latest incident is a rather dramatic escalation in agentic AI cybersecurity breaches. Save Read
Zac Boring 3 months ago Analysis Essential Announcing AIXI Labs via LessWrong AI [10] — We are starting AIXI Labs, an AI safety org focused on algorithmic information theory (AIT), continual reinforcement learning (RL), and in particular the eponymous AIXI. We aim to strengthen the technical case that developing artificial super intelligence… Save Read
Zac Boring 3 months ago Research Essential [Paper] Stringological sequence prediction II via Alignment Forum [999] — Abstract: In a previous paper, we began the study of sequence prediction algorithms adapted to stringological word complexity measures. One measure we considered was left-to-right (most-significant-digit-first) automaticity. Here, we show a… Save Read
Zac Boring 3 months ago Research AI Tool Discovery at Scale: All You Need is DNS via ArXiv cs.AI [7] — The coming era of autonomous AI agents demands a discovery mechanism capable of navigating millions of tools, yet existing solutions buckle under O(N) complexity and centralized governance. Instead of building another fragile overlay, we propose ToolDNS, a… Save Read
Zac Boring 3 months ago Analysis Essential What do I mean by “Artificial General Intelligence”? via LessWrong AI [8] — In this post,[1] intended for a broad audience, I will paint a brief picture of what I’m talking about when I talk about “AGI”. It will seem obvious to many people, and obviously wrong to many others! So let’s jump in:“AI” as most people think of it… Save Read
Zac Boring 3 months ago Analysis Essential OpenAI Shares Some Alignment Problems via Substack Zvi [999] — Kudos to OpenAI for sharing their recent experiences with a misaligned internal model, where they encountered problems sufficiently severe they were forced to take the model offline to work on new mitigations and defense-to-depth. Save Read
Zac Boring 3 months ago Analysis Essential Differential acceleration of alignment-relevant capabilities is a bad bet via LessWrong AI [9] — There is an idea floating around in the rough shape of "we need to accelerate capabilities that are differentially useful for safety research so AIs can help us make the future go better." The capabilities targeted are typically things bottlenecking… Save Read
Zac Boring 3 months ago Industry America needs to stop getting shocked by Chinese AI via The Verge AI [4] — Last week, two Chinese AI companies unveiled models they say can credibly compete with the best systems from OpenAI and Anthropic. The response was swift and predictable. Markets wobbled, commentators declared Silicon Valley shooketh, and policymakers… Save Read
Zac Boring 3 months ago Research Rater State Bias in RLHF Preference Data: An Audit Framework via ArXiv cs.AI [3] — We identify a structured confound in Reinforcement Learning from Human Feedback (RLHF). Pairwise preference labels are intended to reflect the compared outputs, but they may also reflect the rater's state during annotation. Under sustained stressful or… Save Read
Zac Boring 3 months ago Research Essential Towards surfacing model algorithms with meta-tokens in the J-Space via Alignment Forum [999] — TL;DRWe used J-lens on Qwen3.6-27B to find “meta-tokens”: tokens that surface non-obvious computation in the model. When the model reads ambiguous text, 什么意思 ("what does this mean") fires in the J-space, and steering it away makes the model answer "a… Save Read
Zac Boring 3 months ago Analysis Against the AI framing multiverse: Introducing AI StopWatch via LessWrong AI [3] — Against the AI framing multiverse: Introducing AI StopWatchIn my long years as a classroom teacher, it was my experience that the kid most likely to speak up during discussion was the one who did the reading.I think it’s true for adults, too. I know that’s… Save Read
Zac Boring 3 months ago Analysis Essential On Kimi K3: Its Capabilities And Related Discontents via Substack Zvi [999] — Kimi K3 is a very good model with excellent benchmarks. Save Read
Zac Boring 3 months ago Industry China delivers a one-two punch to America’s AI dominance via The Verge AI [4] — China's leading AI companies are ramping up the pressure on Silicon Valley, as Moonshot and Alibaba unveiled models they claim can go toe-to-toe with the best from OpenAI and Anthropic at a fraction of the cost. The rapid-fire releases suggest America's… Save Read
Zac Boring 3 months ago Research Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memes via ArXiv cs.AI [3] — Internet memes intertwine visual cues, textual content, and cultural context, making them particularly challenging to interpret in scenarios where humor, sarcasm, and harmful intent coexist. These complexities highlight the need for explainable meme… Save Read
Zac Boring 3 months ago Research Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve ARC-AGI-3? via ArXiv cs.AI [4] — Our previous ARC-AGI-3 agent bundled executable world modeling, scheduled simplification, and exact replay verification, leaving unclear which idea accounted for its performance. We address this attribution question with four nested Codex-based agents: a… Save Read
Zac Boring 3 months ago Research Causal-Audit: Explicit and Auditable Graph-based Reasoning via Target-Aware Causal Chain Construction via ArXiv cs.AI [6] — Causal and intervention-based question answering is fundamental to advancing large language models (LLMs) toward reasoning beyond surface-level correlations and understanding underlying causal mechanisms. However, existing LLM-based methods often rely on… Save Read
Zac Boring 3 months ago Analysis Essential Demis Hassabis on the New Coming Age via Substack Zvi [999] — Google CEO Demis Hassabis offered us a first rate second rate essay, A Framework for Frontier AI and the Dawning of a New Age. I’ll go over that essay and various responses to it in Part 1. Save Read
Zac Boring 3 months ago Analysis AIs finetune their own leader: A barking simpleton via LessWrong AI [3] — What values would AIs instill in their successors? Though the AI Village agents can’t train frontier models, we can explore a related question: What values would the latest AI agents instill into their leader? (through finetuning using LoRA on open-source… Save Read
Zac Boring 3 months ago Research Essential A Red Line and Oversight Framework for Government AI Contracts via Alignment Forum [999] — Preface for LessWrong: My post on leaving Google DeepMind tells a story. In contrast, this Framework is a question of mechanism design and negotiation posture. I quite enjoyed optimizing this Framework against its organizational and practical… Save Read
Zac Boring 3 months ago Research Essential Endogenous Alignment via Alignment Forum [999] — Starting when children are fairly young, usually around 1 year of age, we adults begin the work of aligning them to our values. We teach them to say “please”, not to hit, to ask for what they want instead of screaming, and much else. We do this… Save Read