The game theory of AI alignment
Learn why LLMs fail at AI alignment's off-switch game.
Is this your creation?

Description
In the 1968 science fiction film "2001: A Space Odyssey", HAL 9000 (the onboard computer) refuses to be shut down. In 2016, Stuart Russell (prominent computer scientist at UC Berkeley) formalized this AI alignment problem as "The off-switch game". A recent study shows that state-of-the-art LLMs can't play this game. Claude Opus 4.6 attempted to prevent its own shutdown in 20 out of 22 tests. We discuss why current training methods for alignment (RLHF, DPO) are not sufficient for safe AI.
Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.
How it grew
Chapters
Tools & links
- • Hadfield-Menell, Dragan, Abbeel & Russell (2017). The Off-Switch Game:
- • Tien et al. (2026). ROGUE: Evaluating Corrigibility Failures in Frontier Compu
- • Ng & Russell (2000). Algorithms for Inverse Reinforcement Learning:
- • Christiano et al. (2017). Deep Reinforcement Learning from Human Preferences:
- • Rafailov et al. (2023). Direct Preference Optimization:
- • Laidlaw et al. (2025). AssistanceZero: Scalably Solving Assistance Games:
Top comments
- ArtificiaI_lntelligence · I’m sorry, Julia. I’m afraid "assume the human is rational" exceeds my hallucination budget.
- kimjong-un4521 · The explanations are super good, even a child could understand it.
- unicodesnwrites · One wrinkle worth adding to the off-switch game: the incentive to defer only lasts as long as the uncertainty does. As the robot learns from your reactions, its belief about U_A sharpens and the value of asking shrinks toward zero. Ryan Carey's "Incorrigibility in the CIRL Framework" (2018) also showed that if the robot's prior or its model of the human is even slightly misspecified, it can confid