Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2601.18113 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910051326754816 |
|---|---|
| author | Kong, Dezhang Wu, Zhuxi Liu, Shiqi Tan, Zhicheng Lu, Kuichen Li, Minghao Liu, Qichen Chu, Shengyu Xu, Zhenhua Liu, Xuan Han, Meng |
| author_facet | Kong, Dezhang Wu, Zhuxi Liu, Shiqi Tan, Zhicheng Lu, Kuichen Li, Minghao Liu, Qichen Chu, Shengyu Xu, Zhenhua Liu, Xuan Han, Meng |
| contents | LLM-based web agents have become increasingly popular for their utility in daily life and work. However, they exhibit critical vulnerabilities when processing malicious URLs: accepting a disguised malicious URL enables subsequent access to unsafe webpages, which can cause severe damage to service providers and users. Despite this risk, no benchmark currently targets this emerging threat. To address this gap, we propose MalURLBench, the first benchmark for evaluating LLMs' vulnerabilities to malicious URLs. MalURLBench contains 61,845 attack instances spanning 10 real-world scenarios and 7 categories of real malicious websites. Experiments with 12 popular LLMs reveal that existing models struggle to detect elaborately disguised malicious URLs. We further identify and analyze key factors that impact attack success rates and propose URLGuard, a lightweight defense module. We believe this work will provide a foundational resource for advancing the security of web agents. Our code is available at https://github.com/JiangYingEr/MalURLBench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_18113 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MalURLBench: A Benchmark Evaluating Agents' Vulnerabilities When Processing Web URLs Kong, Dezhang Wu, Zhuxi Liu, Shiqi Tan, Zhicheng Lu, Kuichen Li, Minghao Liu, Qichen Chu, Shengyu Xu, Zhenhua Liu, Xuan Han, Meng Cryptography and Security Artificial Intelligence LLM-based web agents have become increasingly popular for their utility in daily life and work. However, they exhibit critical vulnerabilities when processing malicious URLs: accepting a disguised malicious URL enables subsequent access to unsafe webpages, which can cause severe damage to service providers and users. Despite this risk, no benchmark currently targets this emerging threat. To address this gap, we propose MalURLBench, the first benchmark for evaluating LLMs' vulnerabilities to malicious URLs. MalURLBench contains 61,845 attack instances spanning 10 real-world scenarios and 7 categories of real malicious websites. Experiments with 12 popular LLMs reveal that existing models struggle to detect elaborately disguised malicious URLs. We further identify and analyze key factors that impact attack success rates and propose URLGuard, a lightweight defense module. We believe this work will provide a foundational resource for advancing the security of web agents. Our code is available at https://github.com/JiangYingEr/MalURLBench. |
| title | MalURLBench: A Benchmark Evaluating Agents' Vulnerabilities When Processing Web URLs |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2601.18113 |