Noise-Aware Differentially Private Regression via Meta-Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909603745234944 |
|---|---|
| author | Räisä, Ossi Markou, Stratis Ashman, Matthew Bruinsma, Wessel P. Tobaben, Marlon Honkela, Antti Turner, Richard E. |
| author_facet | Räisä, Ossi Markou, Stratis Ashman, Matthew Bruinsma, Wessel P. Tobaben, Marlon Honkela, Antti Turner, Richard E. |
| contents | Many high-stakes applications require machine learning models that protect user privacy and provide well-calibrated, accurate predictions. While Differential Privacy (DP) is the gold standard for protecting user privacy, standard DP mechanisms typically significantly impair performance. One approach to mitigating this issue is pre-training models on simulated data before DP learning on the private data. In this work we go a step further, using simulated data to train a meta-learning model that combines the Convolutional Conditional Neural Process (ConvCNP) with an improved functional DP mechanism of Hall et al. [2013] yielding the DPConvCNP. DPConvCNP learns from simulated data how to map private data to a DP predictive model in one forward pass, and then provides accurate, well-calibrated predictions. We compare DPConvCNP with a DP Gaussian Process (GP) baseline with carefully tuned hyperparameters. The DPConvCNP outperforms the GP baseline, especially on non-Gaussian data, yet is much faster at test time and requires less tuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_08569 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Noise-Aware Differentially Private Regression via Meta-Learning Räisä, Ossi Markou, Stratis Ashman, Matthew Bruinsma, Wessel P. Tobaben, Marlon Honkela, Antti Turner, Richard E. Machine Learning Cryptography and Security Many high-stakes applications require machine learning models that protect user privacy and provide well-calibrated, accurate predictions. While Differential Privacy (DP) is the gold standard for protecting user privacy, standard DP mechanisms typically significantly impair performance. One approach to mitigating this issue is pre-training models on simulated data before DP learning on the private data. In this work we go a step further, using simulated data to train a meta-learning model that combines the Convolutional Conditional Neural Process (ConvCNP) with an improved functional DP mechanism of Hall et al. [2013] yielding the DPConvCNP. DPConvCNP learns from simulated data how to map private data to a DP predictive model in one forward pass, and then provides accurate, well-calibrated predictions. We compare DPConvCNP with a DP Gaussian Process (GP) baseline with carefully tuned hyperparameters. The DPConvCNP outperforms the GP baseline, especially on non-Gaussian data, yet is much faster at test time and requires less tuning. |
| title | Noise-Aware Differentially Private Regression via Meta-Learning |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2406.08569 |