Gandalf the Red: Adaptive Security for LLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915424392708096 |
|---|---|
| author | Pfister, Niklas Volhejn, Václav Knott, Manuel Arias, Santiago Bazińska, Julia Bichurin, Mykhailo Commike, Alan Darling, Janet Dienes, Peter Fiedler, Matthew Haber, David Kraft, Matthias Lancini, Marco Mathys, Max Pascual-Ortiz, Damián Podolak, Jakub Romero-López, Adrià Shiarlis, Kyriacos Signer, Andreas Terek, Zsolt Theocharis, Athanasios Timbrell, Daniel Trautwein, Samuel Watts, Samuel Wu, Yun-Han Rojas-Carulla, Mateo |
| author_facet | Pfister, Niklas Volhejn, Václav Knott, Manuel Arias, Santiago Bazińska, Julia Bichurin, Mykhailo Commike, Alan Darling, Janet Dienes, Peter Fiedler, Matthew Haber, David Kraft, Matthias Lancini, Marco Mathys, Max Pascual-Ortiz, Damián Podolak, Jakub Romero-López, Adrià Shiarlis, Kyriacos Signer, Andreas Terek, Zsolt Theocharis, Athanasios Timbrell, Daniel Trautwein, Samuel Watts, Samuel Wu, Yun-Han Rojas-Carulla, Mateo |
| contents | Current evaluations of defenses against prompt attacks in large language model (LLM) applications often overlook two critical factors: the dynamic nature of adversarial behavior and the usability penalties imposed on legitimate users by restrictive defenses. We propose D-SEC (Dynamic Security Utility Threat Model), which explicitly separates attackers from legitimate users, models multi-step interactions, and expresses the security-utility in an optimizable form. We further address the shortcomings in existing evaluations by introducing Gandalf, a crowd-sourced, gamified red-teaming platform designed to generate realistic, adaptive attack. Using Gandalf, we collect and release a dataset of 279k prompt attacks. Complemented by benign user data, our analysis reveals the interplay between security and utility, showing that defenses integrated in the LLM (e.g., system prompts) can degrade usability even without blocking requests. We demonstrate that restricted application domains, defense-in-depth, and adaptive defenses are effective strategies for building secure and useful LLM applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_07927 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Gandalf the Red: Adaptive Security for LLMs Pfister, Niklas Volhejn, Václav Knott, Manuel Arias, Santiago Bazińska, Julia Bichurin, Mykhailo Commike, Alan Darling, Janet Dienes, Peter Fiedler, Matthew Haber, David Kraft, Matthias Lancini, Marco Mathys, Max Pascual-Ortiz, Damián Podolak, Jakub Romero-López, Adrià Shiarlis, Kyriacos Signer, Andreas Terek, Zsolt Theocharis, Athanasios Timbrell, Daniel Trautwein, Samuel Watts, Samuel Wu, Yun-Han Rojas-Carulla, Mateo Machine Learning Artificial Intelligence Computation and Language Cryptography and Security Current evaluations of defenses against prompt attacks in large language model (LLM) applications often overlook two critical factors: the dynamic nature of adversarial behavior and the usability penalties imposed on legitimate users by restrictive defenses. We propose D-SEC (Dynamic Security Utility Threat Model), which explicitly separates attackers from legitimate users, models multi-step interactions, and expresses the security-utility in an optimizable form. We further address the shortcomings in existing evaluations by introducing Gandalf, a crowd-sourced, gamified red-teaming platform designed to generate realistic, adaptive attack. Using Gandalf, we collect and release a dataset of 279k prompt attacks. Complemented by benign user data, our analysis reveals the interplay between security and utility, showing that defenses integrated in the LLM (e.g., system prompts) can degrade usability even without blocking requests. We demonstrate that restricted application domains, defense-in-depth, and adaptive defenses are effective strategies for building secure and useful LLM applications. |
| title | Gandalf the Red: Adaptive Security for LLMs |
| topic | Machine Learning Artificial Intelligence Computation and Language Cryptography and Security |
| url | https://arxiv.org/abs/2501.07927 |