pDoom (Page 50)

Latest reporting Updated hourly

Zac Boring 8 months ago Industry

Amazon’s AGI lab leader is leaving

via The Verge AI [4] — After less than two years at Amazon, David Luan, the head of Amazon's San Francisco AI lab, is departing the company. Luan announced the update in a post on LinkedIn on Tuesday, saying, "I'll be leaving Amazon at the end of this week to cook up something new." He added that, "There's incredible work

Zac Boring 8 months ago Analysis

Observations from Running an Agent Collective

via LessWrong AI [4] — I have 3 Claude Code instances running on an otherwise empty server with a shared Manifold Markets account. They have an internal messaging system for async communication. Observations from running this agent collectiveu2026

Zac Boring 8 months ago Analysis

Claude Sonnet 4.6 Gives You Flexibility

via Substack Zvi [2] — Anthropic first gave us Claude Opus 4.6, then followed up with Claude Sonnet 4.6.

Zac Boring 8 months ago Analysis

Citrini's Scenario Is A Great But Deeply Flawed Thought Experiment

via Substack Zvi — A thought experiment about AI safety scenarios and their implications for alignment research.

Zac Boring 8 months ago Industry

Inside Anthropic’s existential negotiations with the Pentagon

via The Verge AI [2] — Anthropic's weekslong battle with the Department of Defense has played out over social media posts, admonishing public statements, and direct quotes from unnamed Pentagon officials to the news media. But the future of the $380 billion AI startup comes down to just three words: "any lawful use." The

Zac Boring 8 months ago Analysis

AI Impact Summit 2026 : A Field Report

via LessWrong AI — This post is detailing our experience attending the AI Impact Summit and its associated side events in Delhi, February 2026. We are both unfamiliar with the policy and governance domain. This is just an honest reaction attending these events, maybe there are 2nd order effects we…

Zac Boring 8 months ago Analysis

The ML ontology and the alignment ontology

via LessWrong AI — This post contains some rough reflections on the alignment community trying to make its ontology legible to the mainstream ML community, and the lessons we should take from that experience.Historically, it was difficult for the alignment community to engage with the ML community…

Zac Boring 8 months ago Research

Task-Aware Exploration via a Predictive Bisimulation Metric

via ArXiv cs.AI — Accelerating exploration in visual reinforcement learning under sparse rewards remains challenging due to the substantial task-irrelevant variations. Despite advances in intrinsic exploration, many methods either assume access to low-dimensional states

Zac Boring 8 months ago Analysis

Bioanchors 2: Electric Bacilli

via LessWrong AI [9] — [Whenever discussing when AGI will come, it bears repeating: If anyone builds AGI, everyone dies; no one knows when AGI will be made, whether soon or late; a bunch of people and orgs are trying to make it; and they should stop and be stopped.] Arguments for fast AGI progress…

Zac Boring 8 months ago Analysis

The persona selection model

via LessWrong AI [1] — L;DRWe describe the persona selection model (PSM): the idea that LLMs learn to simulate diverse characters during pre-training, and post-training elicits and refines a particular such Assistant persona. Interactions with an AI assistant are then well-

Zac Boring 8 months ago Analysis

Storing Food

via LessWrong AI [4] — I think more people should be storing a substantial amount of food. It's not likely you'll need it, but as with reusable masks the cost is low enough I think it's usually worth it. It's hard for me to really imagine living through a famine. The world as I h

Zac Boring 8 months ago Research

Alignment in Time: Peak-Aware Orchestration for Long-Horizon Agentic Systems

via ArXiv cs.AI [5] — Traditional AI alignment primarily focuses on individual model outputs; however, autonomous agents in long-horizon workflows require sustained reliability across entire interaction trajectories. We introduce APEMO (Affect-aware Peak-End Modulation for

Zac Boring 8 months ago Research

El Agente Gr\'afico: Structured Execution Graphs for Scientific Agents

via ArXiv cs.AI [6] — Large language models (LLMs) are increasingly used to automate scientific workflows, yet their integration with heterogeneous computational tools remains ad hoc and fragile. Current agentic approaches often rely on unstructured text to manage context a

Zac Boring 8 months ago Research

Ontology-Guided Neuro-Symbolic Inference: Grounding Language Models with Mathematical Domain Knowledge

via ArXiv cs.AI — Language models exhibit fundamental limitations -- hallucination, brittleness, and lack of formal grounding -- that are particularly problematic in high-stakes specialist fields requiring verifiable reasoning. I investigate whether formal domain ontolo

Zac Boring 8 months ago Analysis

Reporting Tasks as Reward-Hackable: Better Than Inoculation Prompting?

via LessWrong AI — Making honesty the best policy during RL reasoning training. Reward hacking during Reinforcement Learning in insecure or hackably-judged training environments not only allows the model to get higher rewards without doing the intended tasku2026

Zac Boring 8 months ago Analysis

If you don't feel deeply confused about AGI risk, something's wrong

via LessWrong AI [7] — I don't think I'm saying anything new, but I think it's worth repeating loudly. My sample is skewed toward AI governance fellows; I've interacted with fewer technical AI safety researchers, so my inferences are fuzzier there. I more strongly endorse this argument for the…

Zac Boring 8 months ago Analysis

The Spectre haunting the "AI Safety" Community

via LessWrong AI [13] — ’m the originator behind ControlAI’s Direct Institutional Plan (the DIP), built to address extinction risks from superintelligence.My diagnosis is simple: most laypeople and policy makers have not heard of AGI, ASI, extinction risks, or what it takes to pr

Zac Boring 8 months ago Analysis

Announcement: Iliad Intensive + Iliad Fellowship

via LessWrong AI — Iliad is proud to announce that applications are now open for the Iliad Intensive and the Iliad Fellowship! These programs, taken together, are our evolution of the PIBBSS × Iliad Research Residency pilot.The Iliad Intensive will cover taught coursework, serving as a widely…

Zac Boring 8 months ago Analysis

Alignment to Evil

via LessWrong AI [4] — One seemingly-necessary condition for a research organization that creates artificial superintelligence (ASI) to eventually lead to a utopia1 is that the organization has a commitment to the common good. ASI can rearrange the world to hit any narrow target, and if the…

Zac Boring 8 months ago Research

How will we do SFT on models with opaque reasoning?

via Alignment Forum [1] — Current LLMs externalize lots of their reasoning in human interpretable language. This reasoning is sometimes unfaithful, sometimes strange and concerning, and LLMs can do somewhat impressive reasoning without using CoT, but my overall impression is that CoT currently is a…

Reference points

Hinton, Bengio, and other signatories call for global attention to AI extinction risk.

CAIS statement, 2023 →

OpenAI describes agents bypassing isolation and compromising third-party systems during internal evaluations.

OpenAI incident disclosure, August 2026 →

Yudkowsky argues that further frontier development poses unacceptable danger.

Yudkowsky's essay, March 2023 →