PII-Compass: Guiding LLM training data extraction prompts towards the target PII via grounding
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915458308898816 |
|---|---|
| author | Nakka, Krishna Kanth Frikha, Ahmed Mendes, Ricardo Jiang, Xue Zhou, Xuebing |
| author_facet | Nakka, Krishna Kanth Frikha, Ahmed Mendes, Ricardo Jiang, Xue Zhou, Xuebing |
| contents | The latest and most impactful advances in large models stem from their increased size. Unfortunately, this translates into an improved memorization capacity, raising data privacy concerns. Specifically, it has been shown that models can output personal identifiable information (PII) contained in their training data. However, reported PIII extraction performance varies widely, and there is no consensus on the optimal methodology to evaluate this risk, resulting in underestimating realistic adversaries. In this work, we empirically demonstrate that it is possible to improve the extractability of PII by over ten-fold by grounding the prefix of the manually constructed extraction prompt with in-domain data. Our approach, PII-Compass, achieves phone number extraction rates of 0.92%, 3.9%, and 6.86% with 1, 128, and 2308 queries, respectively, i.e., the phone number of 1 person in 15 is extractable. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_02943 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PII-Compass: Guiding LLM training data extraction prompts towards the target PII via grounding Nakka, Krishna Kanth Frikha, Ahmed Mendes, Ricardo Jiang, Xue Zhou, Xuebing Cryptography and Security Artificial Intelligence Computation and Language Machine Learning The latest and most impactful advances in large models stem from their increased size. Unfortunately, this translates into an improved memorization capacity, raising data privacy concerns. Specifically, it has been shown that models can output personal identifiable information (PII) contained in their training data. However, reported PIII extraction performance varies widely, and there is no consensus on the optimal methodology to evaluate this risk, resulting in underestimating realistic adversaries. In this work, we empirically demonstrate that it is possible to improve the extractability of PII by over ten-fold by grounding the prefix of the manually constructed extraction prompt with in-domain data. Our approach, PII-Compass, achieves phone number extraction rates of 0.92%, 3.9%, and 6.86% with 1, 128, and 2308 queries, respectively, i.e., the phone number of 1 person in 15 is extractable. |
| title | PII-Compass: Guiding LLM training data extraction prompts towards the target PII via grounding |
| topic | Cryptography and Security Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2407.02943 |