Bayesian Pseudo Posterior Mechanism for Differentially Private Machine Learning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910896674045952 |
|---|---|
| author | Chew, Robert Williams, Matthew R. Segarra, Elan A. Preiss, Alexander J. Konet, Amanda Savitsky, Terrance D. |
| author_facet | Chew, Robert Williams, Matthew R. Segarra, Elan A. Preiss, Alexander J. Konet, Amanda Savitsky, Terrance D. |
| contents | Differential privacy (DP) is becoming increasingly important for deployed machine learning applications because it provides strong guarantees for protecting the privacy of individuals whose data is used to train models. However, DP mechanisms commonly used in machine learning tend to struggle on many real world distributions, including highly imbalanced or small labeled training sets. In this work, we propose a new scalable DP mechanism for deep learning models, SWAG-PPM, by using a pseudo posterior distribution that downweights by-record likelihood contributions proportionally to their disclosure risks as the randomized mechanism. As a motivating example from official statistics, we demonstrate SWAG-PPM on a workplace injury text classification task using a highly imbalanced public dataset published by the U.S. Occupational Safety and Health Administration (OSHA). We find that SWAG-PPM exhibits only modest utility degradation against a non-private comparator while greatly outperforming the industry standard DP-SGD for a similar privacy budget. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_21528 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bayesian Pseudo Posterior Mechanism for Differentially Private Machine Learning Chew, Robert Williams, Matthew R. Segarra, Elan A. Preiss, Alexander J. Konet, Amanda Savitsky, Terrance D. Machine Learning Cryptography and Security Differential privacy (DP) is becoming increasingly important for deployed machine learning applications because it provides strong guarantees for protecting the privacy of individuals whose data is used to train models. However, DP mechanisms commonly used in machine learning tend to struggle on many real world distributions, including highly imbalanced or small labeled training sets. In this work, we propose a new scalable DP mechanism for deep learning models, SWAG-PPM, by using a pseudo posterior distribution that downweights by-record likelihood contributions proportionally to their disclosure risks as the randomized mechanism. As a motivating example from official statistics, we demonstrate SWAG-PPM on a workplace injury text classification task using a highly imbalanced public dataset published by the U.S. Occupational Safety and Health Administration (OSHA). We find that SWAG-PPM exhibits only modest utility degradation against a non-private comparator while greatly outperforming the industry standard DP-SGD for a similar privacy budget. |
| title | Bayesian Pseudo Posterior Mechanism for Differentially Private Machine Learning |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2503.21528 |