Preface to the Computational Agent Theory and ASI Alignment Sequence
Key takeaway
The idea of “artificial superintelligence [1] ” (ASI) has recently crossed from the realm of science fiction (in the mainstream perception) to the realm of near-term technological goals openly pursued by most leading commercial organizations in the field of…
Why it's on PDOOM
PDOOM selected this story for its alignment & control signals: Alignment, Superintelligence, Existential Risk.
AlignmentSuperintelligenceExistential RiskFrom Alignment Forum
The idea of “artificial superintelligence [1] ” (ASI) has recently crossed from the realm of science fiction (in the mainstream perception) to the realm of near-term technological goals openly pursued by most leading commercial organizations in the field of AI. Unfortunately, ASI poses a grave existential risk to the human species. This was argued by Eliezer Yudkowsky since 2001 [2] , and by many others (see e.g., Omohundro 2009, Bostrom 2014, Armstrong 2014, Russell 2019, Bengio et al. 2024 (25
By Vanessa Kosoy