LLMsPark: A Benchmark for Evaluating Large Language Models in Strategic Gaming Contexts
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908549275189248 |
|---|---|
| author | Chen, Junhao Sun, Jingbo Li, Xiang Xin, Haidong Xue, Yuhao Xu, Yibin Zhao, Hao |
| author_facet | Chen, Junhao Sun, Jingbo Li, Xiang Xin, Haidong Xue, Yuhao Xu, Yibin Zhao, Hao |
| contents | As large language models (LLMs) advance across diverse tasks, the need for comprehensive evaluation beyond single metrics becomes increasingly important. To fully assess LLM intelligence, it is crucial to examine their interactive dynamics and strategic behaviors. We present LLMsPark, a game theory-based evaluation platform that measures LLMs' decision-making strategies and social behaviors in classic game-theoretic settings, providing a multi-agent environment to explore strategic depth. Our system cross-evaluates 15 leading LLMs (both commercial and open-source) using leaderboard rankings and scoring mechanisms. Higher scores reflect stronger reasoning and strategic capabilities, revealing distinct behavioral patterns and performance differences across models. This work introduces a novel perspective for evaluating LLMs' strategic intelligence, enriching existing benchmarks and broadening their assessment in interactive, game-theoretic scenarios. The benchmark and rankings are publicly available at https://llmsparks.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_16610 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLMsPark: A Benchmark for Evaluating Large Language Models in Strategic Gaming Contexts Chen, Junhao Sun, Jingbo Li, Xiang Xin, Haidong Xue, Yuhao Xu, Yibin Zhao, Hao Computation and Language As large language models (LLMs) advance across diverse tasks, the need for comprehensive evaluation beyond single metrics becomes increasingly important. To fully assess LLM intelligence, it is crucial to examine their interactive dynamics and strategic behaviors. We present LLMsPark, a game theory-based evaluation platform that measures LLMs' decision-making strategies and social behaviors in classic game-theoretic settings, providing a multi-agent environment to explore strategic depth. Our system cross-evaluates 15 leading LLMs (both commercial and open-source) using leaderboard rankings and scoring mechanisms. Higher scores reflect stronger reasoning and strategic capabilities, revealing distinct behavioral patterns and performance differences across models. This work introduces a novel perspective for evaluating LLMs' strategic intelligence, enriching existing benchmarks and broadening their assessment in interactive, game-theoretic scenarios. The benchmark and rankings are publicly available at https://llmsparks.github.io/. |
| title | LLMsPark: A Benchmark for Evaluating Large Language Models in Strategic Gaming Contexts |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.16610 |