Differentially Private Domain Adaptation with Theoretical Guarantees
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909091751788544 |
|---|---|
| author | Bassily, Raef Cortes, Corinna Mao, Anqi Mohri, Mehryar |
| author_facet | Bassily, Raef Cortes, Corinna Mao, Anqi Mohri, Mehryar |
| contents | In many applications, the labeled data at the learner's disposal is subject to privacy constraints and is relatively limited. To derive a more accurate predictor for the target domain, it is often beneficial to leverage publicly available labeled data from an alternative domain, somewhat close to the target domain. This is the modern problem of supervised domain adaptation from a public source to a private target domain. We present two $(ε, δ)$-differentially private adaptation algorithms for supervised adaptation, for which we make use of a general optimization problem, recently shown to benefit from favorable theoretical learning guarantees. Our first algorithm is designed for regression with linear predictors and shown to solve a convex optimization problem. Our second algorithm is a more general solution for loss functions that may be non-convex but Lipschitz and smooth. While our main objective is a theoretical analysis, we also report the results of several experiments first demonstrating that the non-private versions of our algorithms outperform adaptation baselines and next showing that, for larger values of the target sample size or $ε$, the performance of our private algorithms remains close to that of the non-private formulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_08838 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Differentially Private Domain Adaptation with Theoretical Guarantees Bassily, Raef Cortes, Corinna Mao, Anqi Mohri, Mehryar Machine Learning Cryptography and Security In many applications, the labeled data at the learner's disposal is subject to privacy constraints and is relatively limited. To derive a more accurate predictor for the target domain, it is often beneficial to leverage publicly available labeled data from an alternative domain, somewhat close to the target domain. This is the modern problem of supervised domain adaptation from a public source to a private target domain. We present two $(ε, δ)$-differentially private adaptation algorithms for supervised adaptation, for which we make use of a general optimization problem, recently shown to benefit from favorable theoretical learning guarantees. Our first algorithm is designed for regression with linear predictors and shown to solve a convex optimization problem. Our second algorithm is a more general solution for loss functions that may be non-convex but Lipschitz and smooth. While our main objective is a theoretical analysis, we also report the results of several experiments first demonstrating that the non-private versions of our algorithms outperform adaptation baselines and next showing that, for larger values of the target sample size or $ε$, the performance of our private algorithms remains close to that of the non-private formulation. |
| title | Differentially Private Domain Adaptation with Theoretical Guarantees |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2306.08838 |