Distributionally Robust Coreset Selection under Covariate Shift
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909499058552832 |
|---|---|
| author | Tanaka, Tomonari Hanada, Hiroyuki Yang, Hanting Aoyama, Tatsuya Inatsu, Yu Akahane, Satoshi Okura, Yoshito Hashimoto, Noriaki Murayama, Taro Lee, Hanju Kojima, Shinya Takeuchi, Ichiro |
| author_facet | Tanaka, Tomonari Hanada, Hiroyuki Yang, Hanting Aoyama, Tatsuya Inatsu, Yu Akahane, Satoshi Okura, Yoshito Hashimoto, Noriaki Murayama, Taro Lee, Hanju Kojima, Shinya Takeuchi, Ichiro |
| contents | Coreset selection, which involves selecting a small subset from an existing training dataset, is an approach to reducing training data, and various approaches have been proposed for this method. In practical situations where these methods are employed, it is often the case that the data distributions differ between the development phase and the deployment phase, with the latter being unknown. Thus, it is challenging to select an effective subset of training data that performs well across all deployment scenarios. We therefore propose Distributionally Robust Coreset Selection (DRCS). DRCS theoretically derives an estimate of the upper bound for the worst-case test error, assuming that the future covariate distribution may deviate within a defined range from the training distribution. Furthermore, by selecting instances in a way that suppresses the estimate of the upper bound for the worst-case test error, DRCS achieves distributionally robust training instance selection. This study is primarily applicable to convex training computation, but we demonstrate that it can also be applied to deep learning under appropriate approximations. In this paper, we focus on covariate shift, a type of data distribution shift, and demonstrate the effectiveness of DRCS through experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_14253 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Distributionally Robust Coreset Selection under Covariate Shift Tanaka, Tomonari Hanada, Hiroyuki Yang, Hanting Aoyama, Tatsuya Inatsu, Yu Akahane, Satoshi Okura, Yoshito Hashimoto, Noriaki Murayama, Taro Lee, Hanju Kojima, Shinya Takeuchi, Ichiro Machine Learning Coreset selection, which involves selecting a small subset from an existing training dataset, is an approach to reducing training data, and various approaches have been proposed for this method. In practical situations where these methods are employed, it is often the case that the data distributions differ between the development phase and the deployment phase, with the latter being unknown. Thus, it is challenging to select an effective subset of training data that performs well across all deployment scenarios. We therefore propose Distributionally Robust Coreset Selection (DRCS). DRCS theoretically derives an estimate of the upper bound for the worst-case test error, assuming that the future covariate distribution may deviate within a defined range from the training distribution. Furthermore, by selecting instances in a way that suppresses the estimate of the upper bound for the worst-case test error, DRCS achieves distributionally robust training instance selection. This study is primarily applicable to convex training computation, but we demonstrate that it can also be applied to deep learning under appropriate approximations. In this paper, we focus on covariate shift, a type of data distribution shift, and demonstrate the effectiveness of DRCS through experiments. |
| title | Distributionally Robust Coreset Selection under Covariate Shift |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2501.14253 |