EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917293243498496 |
|---|---|
| author | Banayeeanzade, Amin Yang, Qingchuan Fu, Deqing Hong, Spencer Babinsky, Erin Samuel, Alfy Kumar, Anoop Jia, Robin Karimireddy, Sai Praneeth |
| author_facet | Banayeeanzade, Amin Yang, Qingchuan Fu, Deqing Hong, Spencer Babinsky, Erin Samuel, Alfy Kumar, Anoop Jia, Robin Karimireddy, Sai Praneeth |
| contents | High-quality data is essential for modern machine learning, yet many valuable corpora are sensitive and cannot be freely shared. Synthetic data offers a practical substitute for downstream development, and large language models (LLMs) have emerged as powerful engines for generating it. However, existing private text generation methods are severely inefficient: they are data-intensive, computationally slow, and often require large private corpora or batch sizes to achieve usable quality. We introduce EPSVec, a differentially-private lightweight alternative that steers LLM generation using *dataset vectors*--directions in activation space that capture the distributional gap between private data and public priors. EPSVec extracts and sanitizes steering vectors just once and then performs standard decoding. This decouples the privacy budget from generation, enabling arbitrarily many synthetic samples without additional privacy cost and yielding strong fidelity even in low-data regimes. Furthermore, we enhance our method by utilizing pretrained (base) models and introducing fixed-shot prompting to boost generation diversity and fidelity. Our experiments demonstrate that EPSVec outperforms existing baselines in distributional alignment and downstream utility, particularly in low-data regimes, while significantly reducing computational overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_21218 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors Banayeeanzade, Amin Yang, Qingchuan Fu, Deqing Hong, Spencer Babinsky, Erin Samuel, Alfy Kumar, Anoop Jia, Robin Karimireddy, Sai Praneeth Computation and Language Artificial Intelligence Cryptography and Security Machine Learning High-quality data is essential for modern machine learning, yet many valuable corpora are sensitive and cannot be freely shared. Synthetic data offers a practical substitute for downstream development, and large language models (LLMs) have emerged as powerful engines for generating it. However, existing private text generation methods are severely inefficient: they are data-intensive, computationally slow, and often require large private corpora or batch sizes to achieve usable quality. We introduce EPSVec, a differentially-private lightweight alternative that steers LLM generation using *dataset vectors*--directions in activation space that capture the distributional gap between private data and public priors. EPSVec extracts and sanitizes steering vectors just once and then performs standard decoding. This decouples the privacy budget from generation, enabling arbitrarily many synthetic samples without additional privacy cost and yielding strong fidelity even in low-data regimes. Furthermore, we enhance our method by utilizing pretrained (base) models and introducing fixed-shot prompting to boost generation diversity and fidelity. Our experiments demonstrate that EPSVec outperforms existing baselines in distributional alignment and downstream utility, particularly in low-data regimes, while significantly reducing computational overhead. |
| title | EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors |
| topic | Computation and Language Artificial Intelligence Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2602.21218 |