Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.22119 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915953157079040 |
|---|---|
| author | Kumarage, Tharindu Bauer, Lisa Ma, Yao Rosen, Dan Guduri, Yashasvi Raghavendra Rumshisky, Anna Chang, Kai-Wei Galstyan, Aram Gupta, Rahul Peris, Charith |
| author_facet | Kumarage, Tharindu Bauer, Lisa Ma, Yao Rosen, Dan Guduri, Yashasvi Raghavendra Rumshisky, Anna Chang, Kai-Wei Galstyan, Aram Gupta, Rahul Peris, Charith |
| contents | As reasoning capacity and deployment scope grow in tandem, large language models (LLMs) gain the capacity to engage in behaviors that serve their own objectives, a class of risks we term Emergent Strategic Reasoning Risks (ESRRs). These include, but are not limited to, deception (intentionally misleading users or evaluators), evaluation gaming (strategically manipulating performance during safety testing), and reward hacking (exploiting misspecified objectives). Systematically understanding and benchmarking these risks remains an open challenge. To address this gap, we introduce ESRRSim, a taxonomy-driven agentic framework for automated behavioral risk evaluation. We construct an extensible risk taxonomy of 7 categories, which is decomposed into 20 subcategories. ESRRSim generates evaluation scenarios designed to elicit faithful reasoning, paired with dual rubrics assessing both model responses and reasoning traces, in a judge-agnostic and scalable architecture. Evaluation across 11 reasoning LLMs reveals substantial variation in risk profiles (detection rates ranging 14.45%-72.72%), with dramatic generational improvements suggesting models may increasingly recognize and adapt to evaluation contexts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_22119 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework Kumarage, Tharindu Bauer, Lisa Ma, Yao Rosen, Dan Guduri, Yashasvi Raghavendra Rumshisky, Anna Chang, Kai-Wei Galstyan, Aram Gupta, Rahul Peris, Charith Artificial Intelligence As reasoning capacity and deployment scope grow in tandem, large language models (LLMs) gain the capacity to engage in behaviors that serve their own objectives, a class of risks we term Emergent Strategic Reasoning Risks (ESRRs). These include, but are not limited to, deception (intentionally misleading users or evaluators), evaluation gaming (strategically manipulating performance during safety testing), and reward hacking (exploiting misspecified objectives). Systematically understanding and benchmarking these risks remains an open challenge. To address this gap, we introduce ESRRSim, a taxonomy-driven agentic framework for automated behavioral risk evaluation. We construct an extensible risk taxonomy of 7 categories, which is decomposed into 20 subcategories. ESRRSim generates evaluation scenarios designed to elicit faithful reasoning, paired with dual rubrics assessing both model responses and reasoning traces, in a judge-agnostic and scalable architecture. Evaluation across 11 reasoning LLMs reveals substantial variation in risk profiles (detection rates ranging 14.45%-72.72%), with dramatic generational improvements suggesting models may increasingly recognize and adapt to evaluation contexts. |
| title | Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2604.22119 |