PriorBoost: An Adaptive Algorithm for Learning from Aggregate Responses
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929237087223808 |
|---|---|
| author | Javanmard, Adel Fahrbach, Matthew Mirrokni, Vahab |
| author_facet | Javanmard, Adel Fahrbach, Matthew Mirrokni, Vahab |
| contents | This work studies algorithms for learning from aggregate responses. We focus on the construction of aggregation sets (called bags in the literature) for event-level loss functions. We prove for linear regression and generalized linear models (GLMs) that the optimal bagging problem reduces to one-dimensional size-constrained $k$-means clustering. Further, we theoretically quantify the advantage of using curated bags over random bags. We then propose the PriorBoost algorithm, which adaptively forms bags of samples that are increasingly homogeneous with respect to (unobserved) individual responses to improve model quality. We study label differential privacy for aggregate learning, and we also provide extensive experiments showing that PriorBoost regularly achieves optimal model quality for event-level predictions, in stark contrast to non-adaptive algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_04987 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PriorBoost: An Adaptive Algorithm for Learning from Aggregate Responses Javanmard, Adel Fahrbach, Matthew Mirrokni, Vahab Machine Learning Data Structures and Algorithms This work studies algorithms for learning from aggregate responses. We focus on the construction of aggregation sets (called bags in the literature) for event-level loss functions. We prove for linear regression and generalized linear models (GLMs) that the optimal bagging problem reduces to one-dimensional size-constrained $k$-means clustering. Further, we theoretically quantify the advantage of using curated bags over random bags. We then propose the PriorBoost algorithm, which adaptively forms bags of samples that are increasingly homogeneous with respect to (unobserved) individual responses to improve model quality. We study label differential privacy for aggregate learning, and we also provide extensive experiments showing that PriorBoost regularly achieves optimal model quality for event-level predictions, in stark contrast to non-adaptive algorithms. |
| title | PriorBoost: An Adaptive Algorithm for Learning from Aggregate Responses |
| topic | Machine Learning Data Structures and Algorithms |
| url | https://arxiv.org/abs/2402.04987 |