Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917944441700352 |
|---|---|
| author | Zhan, Qiusi Fang, Richard Panchal, Henil Shalin Kang, Daniel |
| author_facet | Zhan, Qiusi Fang, Richard Panchal, Henil Shalin Kang, Daniel |
| contents | Large Language Model (LLM) agents exhibit remarkable performance across diverse applications by using external tools to interact with environments. However, integrating external tools introduces security risks, such as indirect prompt injection (IPI) attacks. Despite defenses designed for IPI attacks, their robustness remains questionable due to insufficient testing against adaptive attacks. In this paper, we evaluate eight different defenses and bypass all of them using adaptive attacks, consistently achieving an attack success rate of over 50%. This reveals critical vulnerabilities in current defenses. Our research underscores the need for adaptive attack evaluation when designing defenses to ensure robustness and reliability. The code is available at https://github.com/uiuc-kang-lab/AdaptiveAttackAgent. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_00061 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents Zhan, Qiusi Fang, Richard Panchal, Henil Shalin Kang, Daniel Cryptography and Security Machine Learning Large Language Model (LLM) agents exhibit remarkable performance across diverse applications by using external tools to interact with environments. However, integrating external tools introduces security risks, such as indirect prompt injection (IPI) attacks. Despite defenses designed for IPI attacks, their robustness remains questionable due to insufficient testing against adaptive attacks. In this paper, we evaluate eight different defenses and bypass all of them using adaptive attacks, consistently achieving an attack success rate of over 50%. This reveals critical vulnerabilities in current defenses. Our research underscores the need for adaptive attack evaluation when designing defenses to ensure robustness and reliability. The code is available at https://github.com/uiuc-kang-lab/AdaptiveAttackAgent. |
| title | Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents |
| topic | Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2503.00061 |