HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908591729934336 |
|---|---|
| author | Ren, Xiaoxue Jiang, Penghao Li, Kaixin Huang, Zhiyong Du, Xiaoning Jiang, Jiaojiao Xing, Zhenchang Sun, Jiamou Zhuo, Terry Yue |
| author_facet | Ren, Xiaoxue Jiang, Penghao Li, Kaixin Huang, Zhiyong Du, Xiaoning Jiang, Jiaojiao Xing, Zhenchang Sun, Jiamou Zhuo, Terry Yue |
| contents | Web applications are prime targets for cyberattacks as gateways to critical services and sensitive data. Traditional penetration testing is costly and expertise-intensive, making it difficult to scale with the growing web ecosystem. While language model agents show promise in cybersecurity, modern web applications demand visual understanding, dynamic content handling, and multi-step interactions that only computer-use agents (CUAs) can perform. Yet, their ability to discover and exploit vulnerabilities through graphical interfaces remains largely unexplored. We present HackWorld, the first framework for systematically evaluating CUAs' capabilities to exploit web application vulnerabilities via visual interaction. Unlike sanitized benchmarks, HackWorld includes 36 real-world applications across 11 frameworks and 7 languages, featuring realistic flaws such as injection vulnerabilities, authentication bypasses, and unsafe input handling. Using a Capture-the-Flag (CTF) setup, it tests CUAs' capacity to identify and exploit these weaknesses while navigating complex web interfaces. Evaluation of state-of-the-art CUAs reveals concerning trends: exploitation rates below 12% and low cybersecurity awareness. CUAs often fail at multi-step attack planning and misuse security tools. These results expose the current limitations of CUAs in web security contexts and highlight opportunities for developing more security-aware agents capable of effective vulnerability detection and exploitation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_12200 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities Ren, Xiaoxue Jiang, Penghao Li, Kaixin Huang, Zhiyong Du, Xiaoning Jiang, Jiaojiao Xing, Zhenchang Sun, Jiamou Zhuo, Terry Yue Cryptography and Security Computation and Language Web applications are prime targets for cyberattacks as gateways to critical services and sensitive data. Traditional penetration testing is costly and expertise-intensive, making it difficult to scale with the growing web ecosystem. While language model agents show promise in cybersecurity, modern web applications demand visual understanding, dynamic content handling, and multi-step interactions that only computer-use agents (CUAs) can perform. Yet, their ability to discover and exploit vulnerabilities through graphical interfaces remains largely unexplored. We present HackWorld, the first framework for systematically evaluating CUAs' capabilities to exploit web application vulnerabilities via visual interaction. Unlike sanitized benchmarks, HackWorld includes 36 real-world applications across 11 frameworks and 7 languages, featuring realistic flaws such as injection vulnerabilities, authentication bypasses, and unsafe input handling. Using a Capture-the-Flag (CTF) setup, it tests CUAs' capacity to identify and exploit these weaknesses while navigating complex web interfaces. Evaluation of state-of-the-art CUAs reveals concerning trends: exploitation rates below 12% and low cybersecurity awareness. CUAs often fail at multi-step attack planning and misuse security tools. These results expose the current limitations of CUAs in web security contexts and highlight opportunities for developing more security-aware agents capable of effective vulnerability detection and exploitation. |
| title | HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities |
| topic | Cryptography and Security Computation and Language |
| url | https://arxiv.org/abs/2510.12200 |