Reweighting Improves Conditional Risk Bounds
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912177574641664 |
|---|---|
| author | Zhang, Yikai Lin, Jiahe Li, Fengpei Zheng, Songzhu Raj, Anant Schneider, Anderson Nevmyvaka, Yuriy |
| author_facet | Zhang, Yikai Lin, Jiahe Li, Fengpei Zheng, Songzhu Raj, Anant Schneider, Anderson Nevmyvaka, Yuriy |
| contents | In this work, we study the weighted empirical risk minimization (weighted ERM) schema, in which an additional data-dependent weight function is incorporated when the empirical risk function is being minimized. We show that under a general ``balanceable" Bernstein condition, one can design a weighted ERM estimator to achieve superior performance in certain sub-regions over the one obtained from standard ERM, and the superiority manifests itself through a data-dependent constant term in the error bound. These sub-regions correspond to large-margin ones in classification settings and low-variance ones in heteroscedastic regression settings, respectively. Our findings are supported by evidence from synthetic data experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_02353 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reweighting Improves Conditional Risk Bounds Zhang, Yikai Lin, Jiahe Li, Fengpei Zheng, Songzhu Raj, Anant Schneider, Anderson Nevmyvaka, Yuriy Machine Learning G.3; I.3 In this work, we study the weighted empirical risk minimization (weighted ERM) schema, in which an additional data-dependent weight function is incorporated when the empirical risk function is being minimized. We show that under a general ``balanceable" Bernstein condition, one can design a weighted ERM estimator to achieve superior performance in certain sub-regions over the one obtained from standard ERM, and the superiority manifests itself through a data-dependent constant term in the error bound. These sub-regions correspond to large-margin ones in classification settings and low-variance ones in heteroscedastic regression settings, respectively. Our findings are supported by evidence from synthetic data experiments. |
| title | Reweighting Improves Conditional Risk Bounds |
| topic | Machine Learning G.3; I.3 |
| url | https://arxiv.org/abs/2501.02353 |