SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911462062030848 |
|---|---|
| author | Guo, Longjie Yuan, Chenjie Zhong, Mingyuan Wolfe, Robert Zhong, Ruican Xu, Yue Wen, Bingbing Shen, Hua Wang, Lucy Lu Hiniker, Alexis |
| author_facet | Guo, Longjie Yuan, Chenjie Zhong, Mingyuan Wolfe, Robert Zhong, Ruican Xu, Yue Wen, Bingbing Shen, Hua Wang, Lucy Lu Hiniker, Alexis |
| contents | As LLM-based computer-use agents (CUAs) begin to autonomously interact with real-world interfaces, understanding their vulnerability to manipulative interface designs becomes increasingly critical. We introduce SusBench, an online benchmark for evaluating the susceptibility of CUAs to UI dark patterns, designs that aim to manipulate or deceive users into taking unintentional actions. Drawing nine common dark pattern types from existing taxonomies, we developed a method for constructing believable dark patterns on real-world consumer websites through code injections, and designed 313 evaluation tasks across 55 websites. Our study with 29 participants showed that humans perceived our dark pattern injections to be highly realistic, with the vast majority of participants not noticing that these had been injected by the research team. We evaluated five state-of-the-art CUAs on the benchmark. We found that both human participants and agents are particularly susceptible to the dark patterns of Preselection, Trick Wording, and Hidden Information, while being resilient to other overt dark patterns. Our findings inform the development of more trustworthy CUAs, their use as potential human proxies in evaluating deceptive designs, and the regulation of an online environment increasingly navigated by autonomous agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_11035 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agents Guo, Longjie Yuan, Chenjie Zhong, Mingyuan Wolfe, Robert Zhong, Ruican Xu, Yue Wen, Bingbing Shen, Hua Wang, Lucy Lu Hiniker, Alexis Human-Computer Interaction As LLM-based computer-use agents (CUAs) begin to autonomously interact with real-world interfaces, understanding their vulnerability to manipulative interface designs becomes increasingly critical. We introduce SusBench, an online benchmark for evaluating the susceptibility of CUAs to UI dark patterns, designs that aim to manipulate or deceive users into taking unintentional actions. Drawing nine common dark pattern types from existing taxonomies, we developed a method for constructing believable dark patterns on real-world consumer websites through code injections, and designed 313 evaluation tasks across 55 websites. Our study with 29 participants showed that humans perceived our dark pattern injections to be highly realistic, with the vast majority of participants not noticing that these had been injected by the research team. We evaluated five state-of-the-art CUAs on the benchmark. We found that both human participants and agents are particularly susceptible to the dark patterns of Preselection, Trick Wording, and Hidden Information, while being resilient to other overt dark patterns. Our findings inform the development of more trustworthy CUAs, their use as potential human proxies in evaluating deceptive designs, and the regulation of an online environment increasingly navigated by autonomous agents. |
| title | SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agents |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2510.11035 |