Planning in a recurrent neural network that plays Sokoban
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912402839175168 |
|---|---|
| author | Taufeeque, Mohammad Quirke, Philip Li, Maximilian Cundy, Chris Tucker, Aaron David Gleave, Adam Garriga-Alonso, Adrià |
| author_facet | Taufeeque, Mohammad Quirke, Philip Li, Maximilian Cundy, Chris Tucker, Aaron David Gleave, Adam Garriga-Alonso, Adrià |
| contents | Planning is essential for solving complex tasks, yet the internal mechanisms underlying planning in neural networks remain poorly understood. Building on prior work, we analyze a recurrent neural network (RNN) trained on Sokoban, a challenging puzzle requiring sequential, irreversible decisions. We find that the RNN has a causal plan representation which predicts its future actions about 50 steps in advance. The quality and length of the represented plan increases over the first few steps. We uncover a surprising behavior: the RNN "paces" in cycles to give itself extra computation at the start of a level, and show that this behavior is incentivized by training. Leveraging these insights, we extend the trained RNN to significantly larger, out-of-distribution Sokoban puzzles, demonstrating robust representations beyond the training regime. We open-source our model and code, and believe the neural network's interesting behavior makes it an excellent model organism to deepen our understanding of learned planning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_15421 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Planning in a recurrent neural network that plays Sokoban Taufeeque, Mohammad Quirke, Philip Li, Maximilian Cundy, Chris Tucker, Aaron David Gleave, Adam Garriga-Alonso, Adrià Machine Learning Artificial Intelligence Planning is essential for solving complex tasks, yet the internal mechanisms underlying planning in neural networks remain poorly understood. Building on prior work, we analyze a recurrent neural network (RNN) trained on Sokoban, a challenging puzzle requiring sequential, irreversible decisions. We find that the RNN has a causal plan representation which predicts its future actions about 50 steps in advance. The quality and length of the represented plan increases over the first few steps. We uncover a surprising behavior: the RNN "paces" in cycles to give itself extra computation at the start of a level, and show that this behavior is incentivized by training. Leveraging these insights, we extend the trained RNN to significantly larger, out-of-distribution Sokoban puzzles, demonstrating robust representations beyond the training regime. We open-source our model and code, and believe the neural network's interesting behavior makes it an excellent model organism to deepen our understanding of learned planning. |
| title | Planning in a recurrent neural network that plays Sokoban |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2407.15421 |