On the Privacy Risk of In-context Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913579706351616 |
|---|---|
| author | Duan, Haonan Dziedzic, Adam Yaghini, Mohammad Papernot, Nicolas Boenisch, Franziska |
| author_facet | Duan, Haonan Dziedzic, Adam Yaghini, Mohammad Papernot, Nicolas Boenisch, Franziska |
| contents | Large language models (LLMs) are excellent few-shot learners. They can perform a wide variety of tasks purely based on natural language prompts provided to them. These prompts contain data of a specific downstream task -- often the private dataset of a party, e.g., a company that wants to leverage the LLM for their purposes. We show that deploying prompted models presents a significant privacy risk for the data used within the prompt by instantiating a highly effective membership inference attack. We also observe that the privacy risk of prompted models exceeds fine-tuned models at the same utility levels. After identifying the model's sensitivity to their prompts -- in the form of a significantly higher prediction confidence on the prompted data -- as a cause for the increased risk, we propose ensembling as a mitigation strategy. By aggregating over multiple different versions of a prompted model, membership inference risk can be decreased. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_10512 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On the Privacy Risk of In-context Learning Duan, Haonan Dziedzic, Adam Yaghini, Mohammad Papernot, Nicolas Boenisch, Franziska Machine Learning Cryptography and Security Large language models (LLMs) are excellent few-shot learners. They can perform a wide variety of tasks purely based on natural language prompts provided to them. These prompts contain data of a specific downstream task -- often the private dataset of a party, e.g., a company that wants to leverage the LLM for their purposes. We show that deploying prompted models presents a significant privacy risk for the data used within the prompt by instantiating a highly effective membership inference attack. We also observe that the privacy risk of prompted models exceeds fine-tuned models at the same utility levels. After identifying the model's sensitivity to their prompts -- in the form of a significantly higher prediction confidence on the prompted data -- as a cause for the increased risk, we propose ensembling as a mitigation strategy. By aggregating over multiple different versions of a prompted model, membership inference risk can be decreased. |
| title | On the Privacy Risk of In-context Learning |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2411.10512 |