Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915793962270720 |
|---|---|
| author | Qi, Zhisheng Sahu, Utkarsh Ma, Li Han, Haoyu Rossi, Ryan Dernoncourt, Franck Halappanavar, Mahantesh Ahmed, Nesreen Dong, Yushun Zhao, Yue Zhang, Yu Wang, Yu |
| author_facet | Qi, Zhisheng Sahu, Utkarsh Ma, Li Han, Haoyu Rossi, Ryan Dernoncourt, Franck Halappanavar, Mahantesh Ahmed, Nesreen Dong, Yushun Zhao, Yue Zhang, Yu Wang, Yu |
| contents | Retrieval-Augmented Generation (RAG) has become a cornerstone of knowledge-intensive applications, including enterprise chatbots, healthcare assistants, and agentic memory management. However, recent studies show that knowledge-extraction attacks can recover sensitive knowledge-base content through maliciously crafted queries, raising serious concerns about intellectual property theft and privacy leakage. While prior work has explored individual attack and defense techniques, the research landscape remains fragmented, spanning heterogeneous retrieval embeddings, diverse generation models, and evaluations based on non-standardized metrics and inconsistent datasets. To address this gap, we introduce the first systematic benchmark for knowledge-extraction attacks on RAG systems. Our benchmark covers a broad spectrum of attack and defense strategies, representative retrieval embedding models, and both open- and closed-source generators, all evaluated under a unified experimental framework with standardized protocols across multiple datasets. By consolidating the experimental landscape and enabling reproducible, comparable evaluation, this benchmark provides actionable insights and a practical foundation for developing privacy-preserving RAG systems in the face of emerging knowledge extraction threats. Our code is available here. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_09319 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation Qi, Zhisheng Sahu, Utkarsh Ma, Li Han, Haoyu Rossi, Ryan Dernoncourt, Franck Halappanavar, Mahantesh Ahmed, Nesreen Dong, Yushun Zhao, Yue Zhang, Yu Wang, Yu Cryptography and Security Retrieval-Augmented Generation (RAG) has become a cornerstone of knowledge-intensive applications, including enterprise chatbots, healthcare assistants, and agentic memory management. However, recent studies show that knowledge-extraction attacks can recover sensitive knowledge-base content through maliciously crafted queries, raising serious concerns about intellectual property theft and privacy leakage. While prior work has explored individual attack and defense techniques, the research landscape remains fragmented, spanning heterogeneous retrieval embeddings, diverse generation models, and evaluations based on non-standardized metrics and inconsistent datasets. To address this gap, we introduce the first systematic benchmark for knowledge-extraction attacks on RAG systems. Our benchmark covers a broad spectrum of attack and defense strategies, representative retrieval embedding models, and both open- and closed-source generators, all evaluated under a unified experimental framework with standardized protocols across multiple datasets. By consolidating the experimental landscape and enabling reproducible, comparable evaluation, this benchmark provides actionable insights and a practical foundation for developing privacy-preserving RAG systems in the face of emerging knowledge extraction threats. Our code is available here. |
| title | Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation |
| topic | Cryptography and Security |
| url | https://arxiv.org/abs/2602.09319 |