MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzles
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909818214678528 |
|---|---|
| author | Ji, Yuheng Tan, Huajie Chi, Cheng Xu, Yijie Zhao, Yuting Zhou, Enshen Lyu, Huaihai Wang, Pengwei Wang, Zhongyuan Zhang, Shanghang Zheng, Xiaolong |
| author_facet | Ji, Yuheng Tan, Huajie Chi, Cheng Xu, Yijie Zhao, Yuting Zhou, Enshen Lyu, Huaihai Wang, Pengwei Wang, Zhongyuan Zhang, Shanghang Zheng, Xiaolong |
| contents | We introduce \textsc{MathSticks}, a benchmark for Visual Symbolic Compositional Reasoning (VSCR), which unifies visual perception, symbolic manipulation, and arithmetic consistency. Each task presents an incorrect matchstick equation that must be corrected by moving one or two sticks under strict conservation rules. The benchmark includes both text-guided and purely visual settings, systematically covering digit scale, move complexity, solution multiplicity, and operator variation, with 1.4M generated instances and a curated test set. Evaluations of 14 vision--language models reveal substantial limitations: closed-source models succeed only on simple cases, open-source models fail in the visual regime, while humans exceed 90\% accuracy. These findings establish \textsc{MathSticks} as a rigorous testbed for advancing compositional reasoning across vision and symbols. Our code and dataset are publicly available at https://github.com/Yuheng2000/MathSticks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_00483 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzles Ji, Yuheng Tan, Huajie Chi, Cheng Xu, Yijie Zhao, Yuting Zhou, Enshen Lyu, Huaihai Wang, Pengwei Wang, Zhongyuan Zhang, Shanghang Zheng, Xiaolong Computer Vision and Pattern Recognition We introduce \textsc{MathSticks}, a benchmark for Visual Symbolic Compositional Reasoning (VSCR), which unifies visual perception, symbolic manipulation, and arithmetic consistency. Each task presents an incorrect matchstick equation that must be corrected by moving one or two sticks under strict conservation rules. The benchmark includes both text-guided and purely visual settings, systematically covering digit scale, move complexity, solution multiplicity, and operator variation, with 1.4M generated instances and a curated test set. Evaluations of 14 vision--language models reveal substantial limitations: closed-source models succeed only on simple cases, open-source models fail in the visual regime, while humans exceed 90\% accuracy. These findings establish \textsc{MathSticks} as a rigorous testbed for advancing compositional reasoning across vision and symbols. Our code and dataset are publicly available at https://github.com/Yuheng2000/MathSticks. |
| title | MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzles |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.00483 |