Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.05511 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914861185761280 |
|---|---|
| author | Schramm, Liam Boularias, Abdeslam |
| author_facet | Schramm, Liam Boularias, Abdeslam |
| contents | Monte Carlo tree search (MCTS) has been successful in a variety of domains, but faces challenges with long-horizon exploration when compared to sampling-based motion planning algorithms like Rapidly-Exploring Random Trees. To address these limitations of MCTS, we derive a tree search algorithm based on policy optimization with state occupancy measure regularization, which we call {\it Volume-MCTS}. We show that count-based exploration and sampling-based motion planning can be derived as approximate solutions to this state occupancy measure regularized objective. We test our method on several robot navigation problems, and find that Volume-MCTS outperforms AlphaZero and displays significantly better long-horizon exploration properties. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_05511 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Provably Efficient Long-Horizon Exploration in Monte Carlo Tree Search through State Occupancy Regularization Schramm, Liam Boularias, Abdeslam Machine Learning Robotics Monte Carlo tree search (MCTS) has been successful in a variety of domains, but faces challenges with long-horizon exploration when compared to sampling-based motion planning algorithms like Rapidly-Exploring Random Trees. To address these limitations of MCTS, we derive a tree search algorithm based on policy optimization with state occupancy measure regularization, which we call {\it Volume-MCTS}. We show that count-based exploration and sampling-based motion planning can be derived as approximate solutions to this state occupancy measure regularized objective. We test our method on several robot navigation problems, and find that Volume-MCTS outperforms AlphaZero and displays significantly better long-horizon exploration properties. |
| title | Provably Efficient Long-Horizon Exploration in Monte Carlo Tree Search through State Occupancy Regularization |
| topic | Machine Learning Robotics |
| url | https://arxiv.org/abs/2407.05511 |