Online Feature Updates Improve Online (Generalized) Label Shift Adaptation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913568194035712 |
|---|---|
| author | Wu, Ruihan Datta, Siddhartha Su, Yi Baby, Dheeraj Wang, Yu-Xiang Weinberger, Kilian Q. |
| author_facet | Wu, Ruihan Datta, Siddhartha Su, Yi Baby, Dheeraj Wang, Yu-Xiang Weinberger, Kilian Q. |
| contents | This paper addresses the prevalent issue of label shift in an online setting with missing labels, where data distributions change over time and obtaining timely labels is challenging. While existing methods primarily focus on adjusting or updating the final layer of a pre-trained classifier, we explore the untapped potential of enhancing feature representations using unlabeled data at test-time. Our novel method, Online Label Shift adaptation with Online Feature Updates (OLS-OFU), leverages self-supervised learning to refine the feature extraction process, thereby improving the prediction model. By carefully designing the algorithm, theoretically OLS-OFU maintains the similar online regret convergence to the results in the literature while taking the improved features into account. Empirically, it achieves substantial improvements over existing methods, which is as significant as the gains existing methods have over the baseline (i.e., without distribution shift adaptations). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_03545 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Online Feature Updates Improve Online (Generalized) Label Shift Adaptation Wu, Ruihan Datta, Siddhartha Su, Yi Baby, Dheeraj Wang, Yu-Xiang Weinberger, Kilian Q. Machine Learning This paper addresses the prevalent issue of label shift in an online setting with missing labels, where data distributions change over time and obtaining timely labels is challenging. While existing methods primarily focus on adjusting or updating the final layer of a pre-trained classifier, we explore the untapped potential of enhancing feature representations using unlabeled data at test-time. Our novel method, Online Label Shift adaptation with Online Feature Updates (OLS-OFU), leverages self-supervised learning to refine the feature extraction process, thereby improving the prediction model. By carefully designing the algorithm, theoretically OLS-OFU maintains the similar online regret convergence to the results in the literature while taking the improved features into account. Empirically, it achieves substantial improvements over existing methods, which is as significant as the gains existing methods have over the baseline (i.e., without distribution shift adaptations). |
| title | Online Feature Updates Improve Online (Generalized) Label Shift Adaptation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.03545 |