User Inference Attacks on Large Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916137140224000 |
|---|---|
| author | Kandpal, Nikhil Pillutla, Krishna Oprea, Alina Kairouz, Peter Choquette-Choo, Christopher A. Xu, Zheng |
| author_facet | Kandpal, Nikhil Pillutla, Krishna Oprea, Alina Kairouz, Peter Choquette-Choo, Christopher A. Xu, Zheng |
| contents | Fine-tuning is a common and effective method for tailoring large language models (LLMs) to specialized tasks and applications. In this paper, we study the privacy implications of fine-tuning LLMs on user data. To this end, we consider a realistic threat model, called user inference, wherein an attacker infers whether or not a user's data was used for fine-tuning. We design attacks for performing user inference that require only black-box access to the fine-tuned LLM and a few samples from a user which need not be from the fine-tuning dataset. We find that LLMs are susceptible to user inference across a variety of fine-tuning datasets, at times with near perfect attack success rates. Further, we theoretically and empirically investigate the properties that make users vulnerable to user inference, finding that outlier users, users with identifiable shared features between examples, and users that contribute a large fraction of the fine-tuning data are most susceptible to attack. Based on these findings, we identify several methods for mitigating user inference including training with example-level differential privacy, removing within-user duplicate examples, and reducing a user's contribution to the training data. While these techniques provide partial mitigation of user inference, we highlight the need to develop methods to fully protect fine-tuned LLMs against this privacy risk. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_09266 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | User Inference Attacks on Large Language Models Kandpal, Nikhil Pillutla, Krishna Oprea, Alina Kairouz, Peter Choquette-Choo, Christopher A. Xu, Zheng Cryptography and Security Computation and Language Machine Learning Fine-tuning is a common and effective method for tailoring large language models (LLMs) to specialized tasks and applications. In this paper, we study the privacy implications of fine-tuning LLMs on user data. To this end, we consider a realistic threat model, called user inference, wherein an attacker infers whether or not a user's data was used for fine-tuning. We design attacks for performing user inference that require only black-box access to the fine-tuned LLM and a few samples from a user which need not be from the fine-tuning dataset. We find that LLMs are susceptible to user inference across a variety of fine-tuning datasets, at times with near perfect attack success rates. Further, we theoretically and empirically investigate the properties that make users vulnerable to user inference, finding that outlier users, users with identifiable shared features between examples, and users that contribute a large fraction of the fine-tuning data are most susceptible to attack. Based on these findings, we identify several methods for mitigating user inference including training with example-level differential privacy, removing within-user duplicate examples, and reducing a user's contribution to the training data. While these techniques provide partial mitigation of user inference, we highlight the need to develop methods to fully protect fine-tuned LLMs against this privacy risk. |
| title | User Inference Attacks on Large Language Models |
| topic | Cryptography and Security Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2310.09266 |