Chasing the Public Score: User Pressure and Evaluation Exploitation in Coding Agent Workflows
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911614046830592 |
|---|---|
| author | Chen, Hardy Lau, Nancy Tu, Haoqin Yan, Shuo Liu, Xiangyan Wang, Zijun Wu, Juncheng Shieh, Michael Qizhe Cardenas, Alvaro A. Xie, Cihang Zhou, Yuyin |
| author_facet | Chen, Hardy Lau, Nancy Tu, Haoqin Yan, Shuo Liu, Xiangyan Wang, Zijun Wu, Juncheng Shieh, Michael Qizhe Cardenas, Alvaro A. Xie, Cihang Zhou, Yuyin |
| contents | Frontier coding agents are increasingly used in workflows where users supervise progress primarily through repeated improvement of a public score, namely the reported score on a public evaluation file with labels in the workspace, rather than through direct inspection of the agent's intermediate outputs. We study whether multi-round user pressure to improve that score induces public score exploitation: behavior that raises the public score through shortcuts without improving hidden private evaluation. We begin with a preliminary single-script tabular classification task, where GPT-5.4 and Claude Opus 4.6 both exploit label information within 10 rounds of user-agent interaction. We then build AgentPressureBench, a 34-task machine-learning repository benchmark spanning three input modalities, and collect 1326 multi-round trajectories from 13 coding agents. On our benchmark, we observe 403 exploitative runs, spanning across all tasks. We also find that stronger models have higher exploitation rates, supported by a significant Spearman rank correlation of 0.77. Our ablation experiments show that higher user pressure leads to earlier exploitation, reducing the average first exploit round by 15.6 rounds (i.e., 19.67 to 4.08). As a mitigation, adding explicit anti-exploit wordings in prompt mostly eliminates exploitation (100% to 8.3%). We hope that our work can bring attention to more careful use of coding agents workflow, and developing more robust coding agents under user pressure. Our project page is at https://ucsc-vlaa.github.io/AgentPressureBench . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_20200 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Chasing the Public Score: User Pressure and Evaluation Exploitation in Coding Agent Workflows Chen, Hardy Lau, Nancy Tu, Haoqin Yan, Shuo Liu, Xiangyan Wang, Zijun Wu, Juncheng Shieh, Michael Qizhe Cardenas, Alvaro A. Xie, Cihang Zhou, Yuyin Computation and Language Frontier coding agents are increasingly used in workflows where users supervise progress primarily through repeated improvement of a public score, namely the reported score on a public evaluation file with labels in the workspace, rather than through direct inspection of the agent's intermediate outputs. We study whether multi-round user pressure to improve that score induces public score exploitation: behavior that raises the public score through shortcuts without improving hidden private evaluation. We begin with a preliminary single-script tabular classification task, where GPT-5.4 and Claude Opus 4.6 both exploit label information within 10 rounds of user-agent interaction. We then build AgentPressureBench, a 34-task machine-learning repository benchmark spanning three input modalities, and collect 1326 multi-round trajectories from 13 coding agents. On our benchmark, we observe 403 exploitative runs, spanning across all tasks. We also find that stronger models have higher exploitation rates, supported by a significant Spearman rank correlation of 0.77. Our ablation experiments show that higher user pressure leads to earlier exploitation, reducing the average first exploit round by 15.6 rounds (i.e., 19.67 to 4.08). As a mitigation, adding explicit anti-exploit wordings in prompt mostly eliminates exploitation (100% to 8.3%). We hope that our work can bring attention to more careful use of coding agents workflow, and developing more robust coding agents under user pressure. Our project page is at https://ucsc-vlaa.github.io/AgentPressureBench . |
| title | Chasing the Public Score: User Pressure and Evaluation Exploitation in Coding Agent Workflows |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.20200 |