Bayes' capacity as a measure for reconstruction attacks in federated learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916295381876736 |
|---|---|
| author | Biswas, Sayan Dras, Mark Faustini, Pedro Fernandes, Natasha McIver, Annabelle Palamidessi, Catuscia Sadeghi, Parastoo |
| author_facet | Biswas, Sayan Dras, Mark Faustini, Pedro Fernandes, Natasha McIver, Annabelle Palamidessi, Catuscia Sadeghi, Parastoo |
| contents | Within the machine learning community, reconstruction attacks are a principal attack of concern and have been identified even in federated learning, which was designed with privacy preservation in mind. In federated learning, it has been shown that an adversary with knowledge of the machine learning architecture is able to infer the exact value of a training element given an observation of the weight updates performed during stochastic gradient descent. In response to these threats, the privacy community recommends the use of differential privacy in the stochastic gradient descent algorithm, termed DP-SGD. However, DP has not yet been formally established as an effective countermeasure against reconstruction attacks. In this paper, we formalise the reconstruction threat model using the information-theoretic framework of quantitative information flow. We show that the Bayes' capacity, related to the Sibson mutual information of order infinity, represents a tight upper bound on the leakage of the DP-SGD algorithm to an adversary interested in performing a reconstruction attack. We provide empirical results demonstrating the effectiveness of this measure for comparing mechanisms against reconstruction threats. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_13569 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Bayes' capacity as a measure for reconstruction attacks in federated learning Biswas, Sayan Dras, Mark Faustini, Pedro Fernandes, Natasha McIver, Annabelle Palamidessi, Catuscia Sadeghi, Parastoo Machine Learning Artificial Intelligence Cryptography and Security Information Theory Within the machine learning community, reconstruction attacks are a principal attack of concern and have been identified even in federated learning, which was designed with privacy preservation in mind. In federated learning, it has been shown that an adversary with knowledge of the machine learning architecture is able to infer the exact value of a training element given an observation of the weight updates performed during stochastic gradient descent. In response to these threats, the privacy community recommends the use of differential privacy in the stochastic gradient descent algorithm, termed DP-SGD. However, DP has not yet been formally established as an effective countermeasure against reconstruction attacks. In this paper, we formalise the reconstruction threat model using the information-theoretic framework of quantitative information flow. We show that the Bayes' capacity, related to the Sibson mutual information of order infinity, represents a tight upper bound on the leakage of the DP-SGD algorithm to an adversary interested in performing a reconstruction attack. We provide empirical results demonstrating the effectiveness of this measure for comparing mechanisms against reconstruction threats. |
| title | Bayes' capacity as a measure for reconstruction attacks in federated learning |
| topic | Machine Learning Artificial Intelligence Cryptography and Security Information Theory |
| url | https://arxiv.org/abs/2406.13569 |