Joe Carlsmith Audio
Audio versions of essays by Joe Carlsmith. Philosophy, futurism, and other topics. Text versions at joecarlsmith.com.
Joe Carlsmith Audio
Two sources of beyond-episode goals (Section 2.2.2 of "Scheming AIs")
•
Joe Carlsmith
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
This is section 2.2.2 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?”
Text of the report here: https://arxiv.org/abs/2311.08379
Summary of the report here: https://joecarlsmith.com/2023/11/15/new-report-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power
Audio summary here: https://joecarlsmithaudio.buzzsprout.com/2034731/13969977-introduction-and-summary-of-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power
2.2.2 Two sources of beyond-episode goals
2.2.2.1 Training-game-independent beyond-episode goals
2.2.2.1.1 Are beyond-episode goals the default?
2.2.2.1.2 How will models think about time?
2.2.2.1.3 The role of “reflection”
2.2.2.1.4 Pushing back on beyond-episode goals using adversarial training
2.2.2.2 Training-game-dependent beyond-episode goals
2.2.2.2.1 Can gradient descent “notice” the benefits of turning a non-schemer into a schemer?
2.2.2.2.2 Is SGD pulling scheming out of models by any means necessary?