Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.07617 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908527585394688 |
|---|---|
| author | Li, Minghui Zhang, Hao Zhang, Yechao Wan, Wei Hu, Shengshan Xiaobing, pei Wang, Jing |
| author_facet | Li, Minghui Zhang, Hao Zhang, Yechao Wan, Wei Hu, Shengshan Xiaobing, pei Wang, Jing |
| contents | Direct Prompt Injection (DPI) attacks pose a critical security threat to Large Language Models (LLMs) due to their low barrier of execution and high potential damage. To address the impracticality of existing white-box/gray-box methods and the poor transferability of black-box methods, we propose an activations-guided prompt injection attack framework. We first construct an Energy-based Model (EBM) using activations from a surrogate model to evaluate the quality of adversarial prompts. Guided by the trained EBM, we employ the token-level Markov Chain Monte Carlo (MCMC) sampling to adaptively optimize adversarial prompts, thereby enabling gradient-free black-box attacks. Experimental results demonstrate our superior cross-model transferability, achieving 49.6% attack success rate (ASR) across five mainstream LLMs and 34.6% improvement over human-crafted prompts, and maintaining 36.6% ASR on unseen task scenarios. Interpretability analysis reveals a correlation between activations and attack effectiveness, highlighting the critical role of semantic patterns in transferable vulnerability exploitation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_07617 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Transferable Direct Prompt Injection via Activation-Guided MCMC Sampling Li, Minghui Zhang, Hao Zhang, Yechao Wan, Wei Hu, Shengshan Xiaobing, pei Wang, Jing Artificial Intelligence Direct Prompt Injection (DPI) attacks pose a critical security threat to Large Language Models (LLMs) due to their low barrier of execution and high potential damage. To address the impracticality of existing white-box/gray-box methods and the poor transferability of black-box methods, we propose an activations-guided prompt injection attack framework. We first construct an Energy-based Model (EBM) using activations from a surrogate model to evaluate the quality of adversarial prompts. Guided by the trained EBM, we employ the token-level Markov Chain Monte Carlo (MCMC) sampling to adaptively optimize adversarial prompts, thereby enabling gradient-free black-box attacks. Experimental results demonstrate our superior cross-model transferability, achieving 49.6% attack success rate (ASR) across five mainstream LLMs and 34.6% improvement over human-crafted prompts, and maintaining 36.6% ASR on unseen task scenarios. Interpretability analysis reveals a correlation between activations and attack effectiveness, highlighting the critical role of semantic patterns in transferable vulnerability exploitation. |
| title | Transferable Direct Prompt Injection via Activation-Guided MCMC Sampling |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2509.07617 |