GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913153660485632 |
|---|---|
| author | Cobben, Pepijn Huang, Xuanqiang Angelo Pham, Thao Amelia Dahlgren, Isabel Zhang, Terry Jingchen Jin, Zhijing |
| author_facet | Cobben, Pepijn Huang, Xuanqiang Angelo Pham, Thao Amelia Dahlgren, Isabel Zhang, Terry Jingchen Jin, Zhijing |
| contents | Frontier AI systems are increasingly capable and deployed in high-stakes multi-agent environments. However, existing AI safety benchmarks largely evaluate single agents, leaving multi-agent risks such as coordination failure and conflict poorly understood. We introduce GT-HarmBench, a benchmark of 1,535 high-stakes scenarios spanning game-theoretic structures such as the Prisoner's Dilemma, Stag Hunt and Chicken. Scenarios are drawn from realistic AI risk contexts in the MIT AI Risk Repository. Across 15 frontier models, agents fail to choose socially beneficial actions in 38% of high-stakes cases, such as military escalation, election manipulation, and medical malpractice. We measure sensitivity to game-theoretic prompt framing and ordering, and analyze reasoning patterns driving failures. We further show that game-theoretic interventions improve socially beneficial outcomes by up to 18%. Our results highlight substantial reliability gaps and provide a broad standardized testbed for studying alignment in multi-agent environments. The benchmark and code are available at https://github.com/causalNLP/gt-harmbench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_12316 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory Cobben, Pepijn Huang, Xuanqiang Angelo Pham, Thao Amelia Dahlgren, Isabel Zhang, Terry Jingchen Jin, Zhijing Artificial Intelligence Computation and Language Computers and Society Computer Science and Game Theory Multiagent Systems Frontier AI systems are increasingly capable and deployed in high-stakes multi-agent environments. However, existing AI safety benchmarks largely evaluate single agents, leaving multi-agent risks such as coordination failure and conflict poorly understood. We introduce GT-HarmBench, a benchmark of 1,535 high-stakes scenarios spanning game-theoretic structures such as the Prisoner's Dilemma, Stag Hunt and Chicken. Scenarios are drawn from realistic AI risk contexts in the MIT AI Risk Repository. Across 15 frontier models, agents fail to choose socially beneficial actions in 38% of high-stakes cases, such as military escalation, election manipulation, and medical malpractice. We measure sensitivity to game-theoretic prompt framing and ordering, and analyze reasoning patterns driving failures. We further show that game-theoretic interventions improve socially beneficial outcomes by up to 18%. Our results highlight substantial reliability gaps and provide a broad standardized testbed for studying alignment in multi-agent environments. The benchmark and code are available at https://github.com/causalNLP/gt-harmbench. |
| title | GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory |
| topic | Artificial Intelligence Computation and Language Computers and Society Computer Science and Game Theory Multiagent Systems |
| url | https://arxiv.org/abs/2602.12316 |