An exponential mechanism based on quadratic approximations for fine-tuning machine learning models with privacy guarantees
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916030577639424 |
|---|---|
| author | Tran, Hoang Ramirez, Jorge Wang, Jiayi Bocchinfuso, Alberto Stanley, Christopher Laiu, M. Paul |
| author_facet | Tran, Hoang Ramirez, Jorge Wang, Jiayi Bocchinfuso, Alberto Stanley, Christopher Laiu, M. Paul |
| contents | Fine-tuning adapts a pretrained machine learning model to a small, sensitive dataset, but this process risks memorizing individual new data points, making the model vulnerable to adversaries who seek to extract sensitive information. In this work, we develop a randomized algorithm based on the exponential mechanism for fine-tuning while ensuring differential privacy. Our key idea is to construct a simple utility function that combines a local quadratic approximation of the pretrained model with information from the new dataset. The resulting exponential mechanism admits exact sampling from a multivariate normal distribution in closed form. We establish theoretical privacy guarantees, sensitivity bounds, and accuracy estimations for our method. We further introduce a random-projection strategy that makes the approach scalable to high-dimensional models. Numerical experiments on the MNIST benchmark and the MIMIC clinical dataset demonstrate competitive performance against existing differentially private fine-tuning techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_20521 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | An exponential mechanism based on quadratic approximations for fine-tuning machine learning models with privacy guarantees Tran, Hoang Ramirez, Jorge Wang, Jiayi Bocchinfuso, Alberto Stanley, Christopher Laiu, M. Paul Machine Learning Cryptography and Security Fine-tuning adapts a pretrained machine learning model to a small, sensitive dataset, but this process risks memorizing individual new data points, making the model vulnerable to adversaries who seek to extract sensitive information. In this work, we develop a randomized algorithm based on the exponential mechanism for fine-tuning while ensuring differential privacy. Our key idea is to construct a simple utility function that combines a local quadratic approximation of the pretrained model with information from the new dataset. The resulting exponential mechanism admits exact sampling from a multivariate normal distribution in closed form. We establish theoretical privacy guarantees, sensitivity bounds, and accuracy estimations for our method. We further introduce a random-projection strategy that makes the approach scalable to high-dimensional models. Numerical experiments on the MNIST benchmark and the MIMIC clinical dataset demonstrate competitive performance against existing differentially private fine-tuning techniques. |
| title | An exponential mechanism based on quadratic approximations for fine-tuning machine learning models with privacy guarantees |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2605.20521 |