Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.20708 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912923102740480 |
|---|---|
| author | Wang, Che Zhang, Fuyao Zhang, Jiaming Zhang, Ziqi Wang, Yinghui Huang, Longtao Gao, Jianbo Chen, Zhong Lim, Wei Yang Bryan |
| author_facet | Wang, Che Zhang, Fuyao Zhang, Jiaming Zhang, Ziqi Wang, Yinghui Huang, Longtao Gao, Jianbo Chen, Zhong Lim, Wei Yang Bryan |
| contents | Large Language Model (LLM) agents are susceptible to Indirect Prompt Injection (IPI) attacks, where malicious instructions in retrieved content hijack the agent's execution. Existing defenses typically rely on strict filtering or refusal mechanisms, which suffer from a critical limitation: over-refusal, prematurely terminating valid agentic workflows. We propose ICON, a probing-to-mitigation framework that neutralizes attacks while preserving task continuity. Our key insight is that IPI attacks leave distinct over-focusing signatures in the latent space. We introduce a Latent Space Trace Prober to detect attacks based on high intensity scores. Subsequently, a Mitigating Rectifier performs surgical attention steering that selectively manipulate adversarial query key dependencies while amplifying task relevant elements to restore the LLM's functional trajectory. Extensive evaluations on multiple backbones show that ICON achieves a competitive 0.4% ASR, matching commercial grade detectors, while yielding a over 50% task utility gain. Furthermore, ICON demonstrates robust Out of Distribution(OOD) generalization and extends effectively to multi-modal agents, establishing a superior balance between security and efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_20708 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correction Wang, Che Zhang, Fuyao Zhang, Jiaming Zhang, Ziqi Wang, Yinghui Huang, Longtao Gao, Jianbo Chen, Zhong Lim, Wei Yang Bryan Artificial Intelligence Cryptography and Security Large Language Model (LLM) agents are susceptible to Indirect Prompt Injection (IPI) attacks, where malicious instructions in retrieved content hijack the agent's execution. Existing defenses typically rely on strict filtering or refusal mechanisms, which suffer from a critical limitation: over-refusal, prematurely terminating valid agentic workflows. We propose ICON, a probing-to-mitigation framework that neutralizes attacks while preserving task continuity. Our key insight is that IPI attacks leave distinct over-focusing signatures in the latent space. We introduce a Latent Space Trace Prober to detect attacks based on high intensity scores. Subsequently, a Mitigating Rectifier performs surgical attention steering that selectively manipulate adversarial query key dependencies while amplifying task relevant elements to restore the LLM's functional trajectory. Extensive evaluations on multiple backbones show that ICON achieves a competitive 0.4% ASR, matching commercial grade detectors, while yielding a over 50% task utility gain. Furthermore, ICON demonstrates robust Out of Distribution(OOD) generalization and extends effectively to multi-modal agents, establishing a superior balance between security and efficiency. |
| title | ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correction |
| topic | Artificial Intelligence Cryptography and Security |
| url | https://arxiv.org/abs/2602.20708 |