Mastering Board Games by External and Internal Planning with Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912388669767680 |
|---|---|
| author | Schultz, John Adamek, Jakub Jusup, Matej Lanctot, Marc Kaisers, Michael Perrin, Sarah Hennes, Daniel Shar, Jeremy Lewis, Cannada Ruoss, Anian Zahavy, Tom Veličković, Petar Prince, Laurel Singh, Satinder Malmi, Eric Tomašev, Nenad |
| author_facet | Schultz, John Adamek, Jakub Jusup, Matej Lanctot, Marc Kaisers, Michael Perrin, Sarah Hennes, Daniel Shar, Jeremy Lewis, Cannada Ruoss, Anian Zahavy, Tom Veličković, Petar Prince, Laurel Singh, Satinder Malmi, Eric Tomašev, Nenad |
| contents | Advancing planning and reasoning capabilities of Large Language Models (LLMs) is one of the key prerequisites towards unlocking their potential for performing reliably in complex and impactful domains. In this paper, we aim to demonstrate this across board games (Chess, Fischer Random / Chess960, Connect Four, and Hex), and we show that search-based planning can yield significant improvements in LLM game-playing strength. We introduce, compare and contrast two major approaches: In external search, the model guides Monte Carlo Tree Search (MCTS) rollouts and evaluations without calls to an external game engine, and in internal search, the model is trained to generate in-context a linearized tree of search and a resulting final choice. Both build on a language model pre-trained on relevant domain knowledge, reliably capturing the transition and value functions in the respective environments, with minimal hallucinations. We evaluate our LLM search implementations against game-specific state-of-the-art engines, showcasing substantial improvements in strength over the base model, and reaching Grandmaster-level performance in chess while operating closer to the human search budget. Our proposed approach, combining search with domain knowledge, is not specific to board games, hinting at more general future applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_12119 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mastering Board Games by External and Internal Planning with Language Models Schultz, John Adamek, Jakub Jusup, Matej Lanctot, Marc Kaisers, Michael Perrin, Sarah Hennes, Daniel Shar, Jeremy Lewis, Cannada Ruoss, Anian Zahavy, Tom Veličković, Petar Prince, Laurel Singh, Satinder Malmi, Eric Tomašev, Nenad Artificial Intelligence Computation and Language Machine Learning Advancing planning and reasoning capabilities of Large Language Models (LLMs) is one of the key prerequisites towards unlocking their potential for performing reliably in complex and impactful domains. In this paper, we aim to demonstrate this across board games (Chess, Fischer Random / Chess960, Connect Four, and Hex), and we show that search-based planning can yield significant improvements in LLM game-playing strength. We introduce, compare and contrast two major approaches: In external search, the model guides Monte Carlo Tree Search (MCTS) rollouts and evaluations without calls to an external game engine, and in internal search, the model is trained to generate in-context a linearized tree of search and a resulting final choice. Both build on a language model pre-trained on relevant domain knowledge, reliably capturing the transition and value functions in the respective environments, with minimal hallucinations. We evaluate our LLM search implementations against game-specific state-of-the-art engines, showcasing substantial improvements in strength over the base model, and reaching Grandmaster-level performance in chess while operating closer to the human search budget. Our proposed approach, combining search with domain knowledge, is not specific to board games, hinting at more general future applications. |
| title | Mastering Board Games by External and Internal Planning with Language Models |
| topic | Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2412.12119 |