S-BDT: Distributed Differentially Private Boosted Decision Trees
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914913725710336 |
|---|---|
| author | Peinemann, Thorsten Kirschte, Moritz Stock, Joshua Cotrini, Carlos Mohammadi, Esfandiar |
| author_facet | Peinemann, Thorsten Kirschte, Moritz Stock, Joshua Cotrini, Carlos Mohammadi, Esfandiar |
| contents | We introduce S-BDT: a novel $(\varepsilon,δ)$-differentially private distributed gradient boosted decision tree (GBDT) learner that improves the protection of single training data points (privacy) while achieving meaningful learning goals, such as accuracy or regression error (utility). S-BDT uses less noise by relying on non-spherical multivariate Gaussian noise, for which we show tight subsampling bounds for privacy amplification and incorporate that into a Rényi filter for individual privacy accounting. We experimentally reach the same utility while saving $50\%$ in terms of epsilon for $\varepsilon \le 0.5$ on the Abalone regression dataset (dataset size $\approx 4K$), saving $30\%$ in terms of epsilon for $\varepsilon \le 0.08$ for the Adult classification dataset (dataset size $\approx 50K$), and saving $30\%$ in terms of epsilon for $\varepsilon\leq0.03$ for the Spambase classification dataset (dataset size $\approx 5K$). Moreover, we show that for situations where a GBDT is learning a stream of data that originates from different subpopulations (non-IID), S-BDT improves the saving of epsilon even further. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_12041 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | S-BDT: Distributed Differentially Private Boosted Decision Trees Peinemann, Thorsten Kirschte, Moritz Stock, Joshua Cotrini, Carlos Mohammadi, Esfandiar Cryptography and Security Machine Learning We introduce S-BDT: a novel $(\varepsilon,δ)$-differentially private distributed gradient boosted decision tree (GBDT) learner that improves the protection of single training data points (privacy) while achieving meaningful learning goals, such as accuracy or regression error (utility). S-BDT uses less noise by relying on non-spherical multivariate Gaussian noise, for which we show tight subsampling bounds for privacy amplification and incorporate that into a Rényi filter for individual privacy accounting. We experimentally reach the same utility while saving $50\%$ in terms of epsilon for $\varepsilon \le 0.5$ on the Abalone regression dataset (dataset size $\approx 4K$), saving $30\%$ in terms of epsilon for $\varepsilon \le 0.08$ for the Adult classification dataset (dataset size $\approx 50K$), and saving $30\%$ in terms of epsilon for $\varepsilon\leq0.03$ for the Spambase classification dataset (dataset size $\approx 5K$). Moreover, we show that for situations where a GBDT is learning a stream of data that originates from different subpopulations (non-IID), S-BDT improves the saving of epsilon even further. |
| title | S-BDT: Distributed Differentially Private Boosted Decision Trees |
| topic | Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2309.12041 |