Research - pDoom (Page 8)

Zac Boring 4 months ago Research

Fusion is not one-size-fits-all: Cross-Modal Representation Alignment for Time-to-Event Modeling

via ArXiv cs.AI [4] — Accurate time-to-event (TTE) prediction from multimodal clinical data remains challenging due to modality imbalance and distribution shift. We introduce a foundation model-driven framework for cross-modal representation alignment between CT imaging and…

Zac Boring 4 months ago Research

Synthetic document finetuning for instilling positive traits

via Alignment Forum [999] — This is the fifth in a series of informal research updates from the Google DeepMind Language Model Interpretability team, in interpretability and adjacent areas. The fourth post can be found here.TLDR: Via adapting the methods of Marks et al and Li et…

Zac Boring 4 months ago Research

Hybrid Open-Ended Tri-Evolution Makes Better Deep Researcher

via ArXiv cs.AI [4] — Deep research and agent evolution serve as de-facto tasks for AI agents in real-world applications toward artificial general intelligence. The former enables autonomous retrieval and integration of information in open-ended environments to tackle open-ended…

Zac Boring 4 months ago Research

Why Do Naive SFT Filters For Safety Properties Fail?

via Alignment Forum [999] — This is the fourth in a series of informal research updates from the Google DeepMind Language Model Interpretability team, in interpretability and adjacent areas. The third post can be found here.Since SFT is the cause for many safety relevant…

Zac Boring 4 months ago Research

SFT Drives Gemini’s Safety Properties

via Alignment Forum [999] — This is the third in a series of informal research updates from the Google DeepMind Language Model Interpretability team, in interpretability and adjacent areas. The second post can be found here.In this short post, we describe a surprising finding:…

Zac Boring 4 months ago Research

Building and evaluating model diffing agents

via Alignment Forum [999] — This is the second in a series of research updates from the Google DeepMind Language Model Interpretability team, in interpretability and adjacent areas. The first post can be found here.TL;DRIt is possible to build extremely simple agents that…

Zac Boring 4 months ago Research

Sympathy for both sides of the egregious misalignment debate

via Alignment Forum [999] — On one side of this debate is Yudkowsky & Soares, who think that (if AI progress continues) we’re on a direct path to egregiously-misaligned, scheming, out-of-control, rogue superintelligence (ASI), not even slightly nice, in the absence of…

Zac Boring 4 months ago Research

From AGI to ASI

via ArXiv cs.AI [8] — Over the last decade, building human-level artificial general intelligence has moved from far-fetched speculation to being a concrete next-decade target for many of the largest AI organisations. Achieving this goal would have profound and far-reaching…

Zac Boring 4 months ago Research

Models May Behave Worse When Eval Aware

via Alignment Forum [999] — This is the first in a series of research updates from the Google DeepMind Language Model Interpretability team, in interpretability and adjacent areas.TL;DRIt's often assumed that models will act more aligned when they can tell they're being…

Zac Boring 4 months ago Research

Position: Hippocampal Explicit Memory Is the Cornerstone for AGI

via ArXiv cs.AI [10] — Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks, raising expectations for Artificial General Intelligence (AGI). This position paper argues that integrating explicit memory is the cornerstone for advancing LLMs…

Zac Boring 4 months ago Research

Sequent: scale and automation for higher confidence in alignment

via Alignment Forum [999] — Alignment is not on trackArtificial superintelligence (ASI) may be developed in the next few years. It is unclear whether alignment is on track to be ready on the same timeframe. At a minimum, the empirical programs at AI labs are unlikely to deliver…

Zac Boring 4 months ago Research

Investing in multi-agent AI safety research

via DeepMind Blog [7] — Google DeepMind and partners announce a $10M funding call for multi-agent safety research.

Zac Boring 4 months ago Research

Tracing Eval-Awareness Emergence Through Training of OLMo 3

via Alignment Forum [999] — TL;DRRecent work from Goodfire & UK AISI – Verbalized Eval Awareness Inflates Measured Safety – shows that newer open-weight models verbalize evaluation-awareness (VEA) more often, and that this inflates measured safety. Between OLMo-3-32B-Think and…

Zac Boring 4 months ago Research

A Mike's-Eye View of ARC's Research

via Alignment Forum [999] — Over the past 15 months or so, ARC's technical agenda has developed quite a bit. The advent of the Matching Sampling Principle (MSP), and ideas like it, has begotten a host of concrete technical problems; progress on those problems has given us more…

Zac Boring 4 months ago Research

Efficient tradeoffs and the safety-usefulness tradeoff model

via Alignment Forum [999] — I often use what I’ll call the “safety-usefulness tradeoff model”, which is: developers face a tradeoff between "safety" and "usefulness" of an AI deployment, and the developer has only limited willingness or ability to sacrifice usefulness for the…

Zac Boring 4 months ago Research

Announcing major new donations, and recapping the 2025 fundraiser

via MIRI [999] — This past December, we ran our first fundraiser in six years, setting an ambitious goal of $6M. We ended up receiving a total of $1.8M from small donors and $1.6M in matching from the Survival and Flourishing Fund (SFF) for a total of $3.4M. We’re incredibly…

Zac Boring 4 months ago Research

My research agenda and work

via Alignment Forum [999] — This is a summary of the work I've done and work I plan to do, and the theories of change and AI progress that motivate my work. I've been working full-time on alignment for three years and change, and thinking about brainlike AGI and its alignment…

Zac Boring 4 months ago Research

How Far Did They Go? The Persuasive Tactics of Covert LLM Agents in a Discontinued Field Experiment

via ArXiv cs.AI [5] — This study analyzes a publicly released dataset from a discontinued field experiment on Reddit's r/ChangeMyView. The intervention, conducted by unknown, external researchers and halted following ethical backlash, involved undisclosed AI-generated accounts…

Zac Boring 4 months ago Research

The Saturation Trap and the Subjectivity of Intervention Timing: Why Affect-Based Triggers and LLM Judges Fail to Time Interventions on Autonomous Agents

via ArXiv cs.AI [4] — As autonomous AI agents move from conversational systems to long-horizon software execution, runtime safety layers that decide when to interrupt an agent have become essential. We study this timing problem using a continuous 18-dimensional…

Zac Boring 5 months ago Research

Announcing the ARC White-Box Estimation Challenge

via Alignment Forum [999] — ARC has teamed up with AIcrowd to launch the ARC White-Box Estimation Challenge, a contest to improve upon our estimation algorithms for random MLPs. The warm-up round begins this week, and later rounds will have a total prize pool of at least…