From Mean to Extreme: Formal Differential Privacy Bounds on the Success of Real-World Data Reconstruction Attacks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909814923198464 |
|---|---|
| author | Riess, Anneliese Schwethelm, Kristian Kaiser, Johannes Mueller, Tamara T. Schnabel, Julia A. Rueckert, Daniel Ziller, Alexander |
| author_facet | Riess, Anneliese Schwethelm, Kristian Kaiser, Johannes Mueller, Tamara T. Schnabel, Julia A. Rueckert, Daniel Ziller, Alexander |
| contents | The gold standard for privacy in machine learning, Differential Privacy (DP), is often interpreted through its guarantees against membership inference. However, translating DP budgets into quantitative protection against the more damaging threat of data reconstruction remains a challenging open problem. Existing theoretical analyses of reconstruction risk are typically based on an "identification" threat model, where an adversary with a candidate set seeks a perfect match. When applied to the realistic threat of "from-scratch" attacks, these bounds can lead to an inefficient privacy-utility trade-off.
This paper bridges this critical gap by deriving the first formal privacy bounds tailored to the mechanics of demonstrated Analytic Gradient Inversion Attacks (AGIAs). We first formalize the optimal from-scratch attack strategy for an adversary with no prior knowledge, showing it reduces to a mean estimation problem. We then derive closed-form, probabilistic bounds on this adversary's success, measured by Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR). Our empirical evaluation confirms these bounds remain tight even when the attack is concealed within large, complex network architectures.
Our work provides a crucial second anchor for risk assessment. By establishing a tight, worst-case bound for the from-scratch threat model, we enable practitioners to assess a "risk corridor" bounded by the identification-based worst case on one side and our from-scratch worst case on the other. This allows for a more holistic, context-aware judgment of privacy risk, empowering practitioners to move beyond abstract budgets toward a principled reasoning framework for calibrating the privacy of their models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_12861 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | From Mean to Extreme: Formal Differential Privacy Bounds on the Success of Real-World Data Reconstruction Attacks Riess, Anneliese Schwethelm, Kristian Kaiser, Johannes Mueller, Tamara T. Schnabel, Julia A. Rueckert, Daniel Ziller, Alexander Machine Learning Cryptography and Security The gold standard for privacy in machine learning, Differential Privacy (DP), is often interpreted through its guarantees against membership inference. However, translating DP budgets into quantitative protection against the more damaging threat of data reconstruction remains a challenging open problem. Existing theoretical analyses of reconstruction risk are typically based on an "identification" threat model, where an adversary with a candidate set seeks a perfect match. When applied to the realistic threat of "from-scratch" attacks, these bounds can lead to an inefficient privacy-utility trade-off. This paper bridges this critical gap by deriving the first formal privacy bounds tailored to the mechanics of demonstrated Analytic Gradient Inversion Attacks (AGIAs). We first formalize the optimal from-scratch attack strategy for an adversary with no prior knowledge, showing it reduces to a mean estimation problem. We then derive closed-form, probabilistic bounds on this adversary's success, measured by Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR). Our empirical evaluation confirms these bounds remain tight even when the attack is concealed within large, complex network architectures. Our work provides a crucial second anchor for risk assessment. By establishing a tight, worst-case bound for the from-scratch threat model, we enable practitioners to assess a "risk corridor" bounded by the identification-based worst case on one side and our from-scratch worst case on the other. This allows for a more holistic, context-aware judgment of privacy risk, empowering practitioners to move beyond abstract budgets toward a principled reasoning framework for calibrating the privacy of their models. |
| title | From Mean to Extreme: Formal Differential Privacy Bounds on the Success of Real-World Data Reconstruction Attacks |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2402.12861 |