Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.20452 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911129342574592 |
|---|---|
| author | Sun, Yidan Schlegel, Viktor Nandakumar, Srinivasan Zahid, Iqra Wu, Yuping Del-Pinto, Warren Nenadic, Goran Lam, Siew-Kei Zhang, Jie Bharath, Anil A |
| author_facet | Sun, Yidan Schlegel, Viktor Nandakumar, Srinivasan Zahid, Iqra Wu, Yuping Del-Pinto, Warren Nenadic, Goran Lam, Siew-Kei Zhang, Jie Bharath, Anil A |
| contents | Generative AI offers transformative potential for high-stakes domains such as healthcare and finance, yet privacy and regulatory barriers hinder the use of real-world data. To address this, differentially private synthetic data generation has emerged as a promising alternative. In this work, we introduce a unified benchmark to systematically evaluate the utility and fidelity of text datasets generated under formal Differential Privacy (DP) guarantees. Our benchmark addresses key challenges in domain-specific benchmarking, including choice of representative data and realistic privacy budgets, accounting for pre-training and a variety of evaluation metrics. We assess state-of-the-art privacy-preserving generation methods across five domain-specific datasets, revealing significant utility and fidelity degradation compared to real data, especially under strict privacy constraints. These findings underscore the limitations of current approaches, outline the need for advanced privacy-preserving data sharing methods and set a precedent regarding their evaluation in realistic scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_20452 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Differentially Private Generation of Domain-Specific Text Sun, Yidan Schlegel, Viktor Nandakumar, Srinivasan Zahid, Iqra Wu, Yuping Del-Pinto, Warren Nenadic, Goran Lam, Siew-Kei Zhang, Jie Bharath, Anil A Machine Learning Artificial Intelligence Cryptography and Security Generative AI offers transformative potential for high-stakes domains such as healthcare and finance, yet privacy and regulatory barriers hinder the use of real-world data. To address this, differentially private synthetic data generation has emerged as a promising alternative. In this work, we introduce a unified benchmark to systematically evaluate the utility and fidelity of text datasets generated under formal Differential Privacy (DP) guarantees. Our benchmark addresses key challenges in domain-specific benchmarking, including choice of representative data and realistic privacy budgets, accounting for pre-training and a variety of evaluation metrics. We assess state-of-the-art privacy-preserving generation methods across five domain-specific datasets, revealing significant utility and fidelity degradation compared to real data, especially under strict privacy constraints. These findings underscore the limitations of current approaches, outline the need for advanced privacy-preserving data sharing methods and set a precedent regarding their evaluation in realistic scenarios. |
| title | Evaluating Differentially Private Generation of Domain-Specific Text |
| topic | Machine Learning Artificial Intelligence Cryptography and Security |
| url | https://arxiv.org/abs/2508.20452 |