Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2506.10264 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911001575686144 |
|---|---|
| author | Yin, Qiyue Xu, Pei Li, Qiaozhe Liu, Shengda Shen, Shengqi Wang, Tong Han, Yihong Zhao, Xiaonan Yang, Likun Cao, Shiyue Qiu, Shiyu Liu, Yuxuan Yu, Shizhao Cui, Lei Yan, Chengxin Sun, Jie Tang, Xiangquan Huang, Kaiqi |
| author_facet | Yin, Qiyue Xu, Pei Li, Qiaozhe Liu, Shengda Shen, Shengqi Wang, Tong Han, Yihong Zhao, Xiaonan Yang, Likun Cao, Shiyue Qiu, Shiyu Liu, Yuxuan Yu, Shizhao Cui, Lei Yan, Chengxin Sun, Jie Tang, Xiangquan Huang, Kaiqi |
| contents | Recent breakthroughs in Large Language Models (LLMs) have led to a qualitative leap in artificial intelligence' s performance on reasoning tasks, particularly demonstrating remarkable capabilities in mathematical, symbolic, and commonsense reasoning. However, as a critical component of advanced human cognition, strategic reasoning, i.e., the ability to assess multi-agent behaviors in dynamic environments, formulate action plans, and adapt strategies, has yet to be systematically evaluated or modeled. To address this gap, this paper introduces WGSR-Bench, the first strategy reasoning benchmark for LLMs using wargame as its evaluation environment. Wargame, a quintessential high-complexity strategic scenario, integrates environmental uncertainty, adversarial dynamics, and non-unique strategic choices, making it an effective testbed for assessing LLMs' capabilities in multi-agent decision-making, intent inference, and counterfactual reasoning. WGSR-Bench designs test samples around three core tasks, i.e., Environmental situation awareness, Opponent risk modeling and Policy generation, which serve as the core S-POE architecture, to systematically assess main abilities of strategic reasoning. Finally, an LLM-based wargame agent is designed to integrate these parts for a comprehensive strategy reasoning assessment. With WGSR-Bench, we hope to assess the strengths and limitations of state-of-the-art LLMs in game-theoretic strategic reasoning and to advance research in large model-driven strategic intelligence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_10264 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | WGSR-Bench: Wargame-based Game-theoretic Strategic Reasoning Benchmark for Large Language Models Yin, Qiyue Xu, Pei Li, Qiaozhe Liu, Shengda Shen, Shengqi Wang, Tong Han, Yihong Zhao, Xiaonan Yang, Likun Cao, Shiyue Qiu, Shiyu Liu, Yuxuan Yu, Shizhao Cui, Lei Yan, Chengxin Sun, Jie Tang, Xiangquan Huang, Kaiqi Artificial Intelligence Recent breakthroughs in Large Language Models (LLMs) have led to a qualitative leap in artificial intelligence' s performance on reasoning tasks, particularly demonstrating remarkable capabilities in mathematical, symbolic, and commonsense reasoning. However, as a critical component of advanced human cognition, strategic reasoning, i.e., the ability to assess multi-agent behaviors in dynamic environments, formulate action plans, and adapt strategies, has yet to be systematically evaluated or modeled. To address this gap, this paper introduces WGSR-Bench, the first strategy reasoning benchmark for LLMs using wargame as its evaluation environment. Wargame, a quintessential high-complexity strategic scenario, integrates environmental uncertainty, adversarial dynamics, and non-unique strategic choices, making it an effective testbed for assessing LLMs' capabilities in multi-agent decision-making, intent inference, and counterfactual reasoning. WGSR-Bench designs test samples around three core tasks, i.e., Environmental situation awareness, Opponent risk modeling and Policy generation, which serve as the core S-POE architecture, to systematically assess main abilities of strategic reasoning. Finally, an LLM-based wargame agent is designed to integrate these parts for a comprehensive strategy reasoning assessment. With WGSR-Bench, we hope to assess the strengths and limitations of state-of-the-art LLMs in game-theoretic strategic reasoning and to advance research in large model-driven strategic intelligence. |
| title | WGSR-Bench: Wargame-based Game-theoretic Strategic Reasoning Benchmark for Large Language Models |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2506.10264 |