How Private is Your Attention? Bridging Privacy with In-Context Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915253472722944 |
|---|---|
| author | Bonnerjee, Soham Wei, Zhen Yeon Asch, Anna Nandy, Sagnik Ghosal, Promit |
| author_facet | Bonnerjee, Soham Wei, Zhen Yeon Asch, Anna Nandy, Sagnik Ghosal, Promit |
| contents | In-context learning (ICL)-the ability of transformer-based models to perform new tasks from examples provided at inference time-has emerged as a hallmark of modern language models. While recent works have investigated the mechanisms underlying ICL, its feasibility under formal privacy constraints remains largely unexplored. In this paper, we propose a differentially private pretraining algorithm for linear attention heads and present the first theoretical analysis of the privacy-accuracy trade-off for ICL in linear regression. Our results characterize the fundamental tension between optimization and privacy-induced noise, formally capturing behaviors observed in private training via iterative methods. Additionally, we show that our method is robust to adversarial perturbations of training prompts, unlike standard ridge regression. All theoretical findings are supported by extensive simulations across diverse settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_16000 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | How Private is Your Attention? Bridging Privacy with In-Context Learning Bonnerjee, Soham Wei, Zhen Yeon Asch, Anna Nandy, Sagnik Ghosal, Promit Machine Learning Artificial Intelligence Computation and Language Cryptography and Security In-context learning (ICL)-the ability of transformer-based models to perform new tasks from examples provided at inference time-has emerged as a hallmark of modern language models. While recent works have investigated the mechanisms underlying ICL, its feasibility under formal privacy constraints remains largely unexplored. In this paper, we propose a differentially private pretraining algorithm for linear attention heads and present the first theoretical analysis of the privacy-accuracy trade-off for ICL in linear regression. Our results characterize the fundamental tension between optimization and privacy-induced noise, formally capturing behaviors observed in private training via iterative methods. Additionally, we show that our method is robust to adversarial perturbations of training prompts, unlike standard ridge regression. All theoretical findings are supported by extensive simulations across diverse settings. |
| title | How Private is Your Attention? Bridging Privacy with In-Context Learning |
| topic | Machine Learning Artificial Intelligence Computation and Language Cryptography and Security |
| url | https://arxiv.org/abs/2504.16000 |