PromptShield: Deployable Detection for Prompt Injection Attacks
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910909458284544 |
|---|---|
| author | Jacob, Dennis Alzahrani, Hend Hu, Zhanhao Alomair, Basel Wagner, David |
| author_facet | Jacob, Dennis Alzahrani, Hend Hu, Zhanhao Alomair, Basel Wagner, David |
| contents | Application designers have moved to integrate large language models (LLMs) into their products. However, many LLM-integrated applications are vulnerable to prompt injections. While attempts have been made to address this problem by building prompt injection detectors, many are not yet suitable for practical deployment. To support research in this area, we introduce PromptShield, a benchmark for training and evaluating deployable prompt injection detectors. Our benchmark is carefully curated and includes both conversational and application-structured data. In addition, we use insights from our curation process to fine-tune a new prompt injection detector that achieves significantly higher performance in the low false positive rate (FPR) evaluation regime compared to prior schemes. Our work suggests that careful curation of training data and larger models can contribute to strong detector performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_15145 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PromptShield: Deployable Detection for Prompt Injection Attacks Jacob, Dennis Alzahrani, Hend Hu, Zhanhao Alomair, Basel Wagner, David Cryptography and Security Application designers have moved to integrate large language models (LLMs) into their products. However, many LLM-integrated applications are vulnerable to prompt injections. While attempts have been made to address this problem by building prompt injection detectors, many are not yet suitable for practical deployment. To support research in this area, we introduce PromptShield, a benchmark for training and evaluating deployable prompt injection detectors. Our benchmark is carefully curated and includes both conversational and application-structured data. In addition, we use insights from our curation process to fine-tune a new prompt injection detector that achieves significantly higher performance in the low false positive rate (FPR) evaluation regime compared to prior schemes. Our work suggests that careful curation of training data and larger models can contribute to strong detector performance. |
| title | PromptShield: Deployable Detection for Prompt Injection Attacks |
| topic | Cryptography and Security |
| url | https://arxiv.org/abs/2501.15145 |