Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2604.23887 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911676729655296 |
|---|---|
| author | Deep, Priyal Emmons, Shane Fox, Amy Bacon, Kyle McAllister, Kelley Ortiz, Peter Flautner, Krisztian |
| author_facet | Deep, Priyal Emmons, Shane Fox, Amy Bacon, Kyle McAllister, Kelley Ortiz, Peter Flautner, Krisztian |
| contents | LLM-powered applications routinely embed secrets in system prompts, yet models can be tricked into revealing them. We built an adaptive attacker that evolves its strategies over hundreds of rounds and tested it against nine defense configurations across more than 20,000 attacks. Every defense that relied on the model to protect itself eventually broke. The only defense that held was output filtering, which checks the model's responses via hardcoded rules in separate application code before they reach the user, achieving zero leaks across 15,000 attacks. These results demonstrate that security boundaries must be enforced in application code, not by the model being attacked. Until such defenses are verified by tools like Swept AI, AI systems handling sensitive operations should be restricted to internal, trusted personnel. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_23887 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Evaluation of Prompt Injection Defenses in Large Language Models Deep, Priyal Emmons, Shane Fox, Amy Bacon, Kyle McAllister, Kelley Ortiz, Peter Flautner, Krisztian Cryptography and Security Artificial Intelligence LLM-powered applications routinely embed secrets in system prompts, yet models can be tricked into revealing them. We built an adaptive attacker that evolves its strategies over hundreds of rounds and tested it against nine defense configurations across more than 20,000 attacks. Every defense that relied on the model to protect itself eventually broke. The only defense that held was output filtering, which checks the model's responses via hardcoded rules in separate application code before they reach the user, achieving zero leaks across 15,000 attacks. These results demonstrate that security boundaries must be enforced in application code, not by the model being attacked. Until such defenses are verified by tools like Swept AI, AI systems handling sensitive operations should be restricted to internal, trusted personnel. |
| title | Evaluation of Prompt Injection Defenses in Large Language Models |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2604.23887 |