Teach LLMs to Phish: Stealing Private Information from Language Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911787715133440 |
|---|---|
| author | Panda, Ashwinee Choquette-Choo, Christopher A. Zhang, Zhengming Yang, Yaoqing Mittal, Prateek |
| author_facet | Panda, Ashwinee Choquette-Choo, Christopher A. Zhang, Zhengming Yang, Yaoqing Mittal, Prateek |
| contents | When large language models are trained on private data, it can be a significant privacy risk for them to memorize and regurgitate sensitive information. In this work, we propose a new practical data extraction attack that we call "neural phishing". This attack enables an adversary to target and extract sensitive or personally identifiable information (PII), e.g., credit card numbers, from a model trained on user data with upwards of 10% attack success rates, at times, as high as 50%. Our attack assumes only that an adversary can insert as few as 10s of benign-appearing sentences into the training dataset using only vague priors on the structure of the user data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_00871 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Teach LLMs to Phish: Stealing Private Information from Language Models Panda, Ashwinee Choquette-Choo, Christopher A. Zhang, Zhengming Yang, Yaoqing Mittal, Prateek Cryptography and Security Artificial Intelligence Computation and Language Machine Learning When large language models are trained on private data, it can be a significant privacy risk for them to memorize and regurgitate sensitive information. In this work, we propose a new practical data extraction attack that we call "neural phishing". This attack enables an adversary to target and extract sensitive or personally identifiable information (PII), e.g., credit card numbers, from a model trained on user data with upwards of 10% attack success rates, at times, as high as 50%. Our attack assumes only that an adversary can insert as few as 10s of benign-appearing sentences into the training dataset using only vague priors on the structure of the user data. |
| title | Teach LLMs to Phish: Stealing Private Information from Language Models |
| topic | Cryptography and Security Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2403.00871 |