Minimax Regret Learning for Data with Heterogeneous Subgroups
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911179856674816 |
|---|---|
| author | Mo, Weibin Tang, Weijing Xue, Songkai Liu, Yufeng Zhu, Ji |
| author_facet | Mo, Weibin Tang, Weijing Xue, Songkai Liu, Yufeng Zhu, Ji |
| contents | Modern complex datasets often consist of various sub-populations with known group information. In the presence of sub-population heterogeneity, it is crucial to develop robust and generalizable learning methods that (1) can enjoy robust performance on each of the training populations, and (2) is generalizable to an unseen testing population. While various min-max formulations have been proposed to achieve (1) in the robust learning literature, their generalization to an unseen testing is less explored. Moreover, a general min-max formulation can be sensitive to the noise heterogeneity, and, in the extreme case, be degenerate such that a single high-noise population dominates. The min-max-regret (MMR) can mitigate these challenges. In this work, we consider a distribution-free robust hierarchical model for the generalization from multiple training populations to an unseen testing population. Under the robust hierarchical model, the empirical MMR can enjoy the regret guarantees on each of the training populations as well as the unseen testing population. We further specialize the general MMR framework to linear regression and generalized linear model, where we characterize the geometry of MMR and its distinction from other robust methods. We demonstrate the effectiveness of MMR through extensive simulation studies and an application to image recognition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_01709 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Minimax Regret Learning for Data with Heterogeneous Subgroups Mo, Weibin Tang, Weijing Xue, Songkai Liu, Yufeng Zhu, Ji Methodology Statistics Theory Machine Learning Modern complex datasets often consist of various sub-populations with known group information. In the presence of sub-population heterogeneity, it is crucial to develop robust and generalizable learning methods that (1) can enjoy robust performance on each of the training populations, and (2) is generalizable to an unseen testing population. While various min-max formulations have been proposed to achieve (1) in the robust learning literature, their generalization to an unseen testing is less explored. Moreover, a general min-max formulation can be sensitive to the noise heterogeneity, and, in the extreme case, be degenerate such that a single high-noise population dominates. The min-max-regret (MMR) can mitigate these challenges. In this work, we consider a distribution-free robust hierarchical model for the generalization from multiple training populations to an unseen testing population. Under the robust hierarchical model, the empirical MMR can enjoy the regret guarantees on each of the training populations as well as the unseen testing population. We further specialize the general MMR framework to linear regression and generalized linear model, where we characterize the geometry of MMR and its distinction from other robust methods. We demonstrate the effectiveness of MMR through extensive simulation studies and an application to image recognition. |
| title | Minimax Regret Learning for Data with Heterogeneous Subgroups |
| topic | Methodology Statistics Theory Machine Learning |
| url | https://arxiv.org/abs/2405.01709 |