Overtly misaligned trajectories score highly in RL.
Key takeaway
It's going to be so embarrassing if we all die due to RL environments rewarding egregiously misaligned behavior.
Why it's on PDOOM
PDOOM selected this story as relevant to risk analysis.
From LessWrong AI
It's going to be so embarrassing if we all die due to RL environments rewarding egregiously misaligned behavior. Like at least let us be killed by misgeneralization. — Thomas KwaConstellation vs MIRI vs Reality Current AI agents often behave overtly egregiously misaligned. By “overt”, I mean that a human, reading the transcript, would say “The agent is obviously acting in direct opposition to the specification, user intent, and any common-sense understanding of good behaviour.” The reason, it se
By Cleo Nardo