PuzzleJAX: A Benchmark for Reasoning and Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912551224213504 |
|---|---|
| author | Earle, Sam Todd, Graham Li, Yuchen Khalifa, Ahmed Nasir, Muhammad Umair Jiang, Zehua Banburski-Fahey, Andrzej Togelius, Julian |
| author_facet | Earle, Sam Todd, Graham Li, Yuchen Khalifa, Ahmed Nasir, Muhammad Umair Jiang, Zehua Banburski-Fahey, Andrzej Togelius, Julian |
| contents | We introduce PuzzleJAX, a GPU-accelerated puzzle game engine and description language designed to support rapid benchmarking of tree search, reinforcement learning, and LLM reasoning abilities. Unlike existing GPU-accelerated learning environments that provide hard-coded implementations of fixed sets of games, PuzzleJAX allows dynamic compilation of any game expressible in its domain-specific language (DSL). This DSL follows PuzzleScript, which is a popular and accessible online game engine for designing puzzle games. In this paper, we validate in PuzzleJAX several hundred of the thousands of games designed in PuzzleScript by both professional designers and casual creators since its release in 2013, thereby demonstrating PuzzleJAX's coverage of an expansive, expressive, and human-relevant space of tasks. By analyzing the performance of search, learning, and language models on these games, we show that PuzzleJAX can naturally express tasks that are both simple and intuitive to understand, yet often deeply challenging to master, requiring a combination of control, planning, and high-level insight. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_16821 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PuzzleJAX: A Benchmark for Reasoning and Learning Earle, Sam Todd, Graham Li, Yuchen Khalifa, Ahmed Nasir, Muhammad Umair Jiang, Zehua Banburski-Fahey, Andrzej Togelius, Julian Artificial Intelligence Machine Learning We introduce PuzzleJAX, a GPU-accelerated puzzle game engine and description language designed to support rapid benchmarking of tree search, reinforcement learning, and LLM reasoning abilities. Unlike existing GPU-accelerated learning environments that provide hard-coded implementations of fixed sets of games, PuzzleJAX allows dynamic compilation of any game expressible in its domain-specific language (DSL). This DSL follows PuzzleScript, which is a popular and accessible online game engine for designing puzzle games. In this paper, we validate in PuzzleJAX several hundred of the thousands of games designed in PuzzleScript by both professional designers and casual creators since its release in 2013, thereby demonstrating PuzzleJAX's coverage of an expansive, expressive, and human-relevant space of tasks. By analyzing the performance of search, learning, and language models on these games, we show that PuzzleJAX can naturally express tasks that are both simple and intuitive to understand, yet often deeply challenging to master, requiring a combination of control, planning, and high-level insight. |
| title | PuzzleJAX: A Benchmark for Reasoning and Learning |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2508.16821 |