Mixed Semi-Supervised Generalized-Linear-Regression with Applications to Deep-Learning and Interpolators
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914159174615040 |
|---|---|
| author | Yuval, Oren Rosset, Saharon |
| author_facet | Yuval, Oren Rosset, Saharon |
| contents | We present a methodology for using unlabeled data to design semi-supervised learning (SSL) methods that improve the predictive performance of supervised learning for regression tasks. The main idea is to design different mechanisms for integrating the unlabeled data, and include in each of them a mixing parameter $α$, controlling the weight given to the unlabeled data. Focusing on Generalized Linear Models (GLM) and linear interpolators classes of models, we analyze the characteristics of different mixing mechanisms, and prove that it is consistently beneficial to integrate the unlabeled data with some nonzero mixing ratio $α>0$, in terms of predictive performance. Moreover, we provide a rigorous framework to estimate the best mixing ratio where mixed-SSL delivers the best predictive performance, while using the labeled and unlabeled data on hand. The effectiveness of our methodology in delivering substantial improvement compared to the standard supervised models, in a variety of settings, is demonstrated empirically through extensive simulation, providing empirical support for our theoretical analysis. We also demonstrate the applicability of our methodology (with some heuristic modifications) to improve more complex models, such as deep neural networks, in real-world regression tasks |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_09526 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Mixed Semi-Supervised Generalized-Linear-Regression with Applications to Deep-Learning and Interpolators Yuval, Oren Rosset, Saharon Methodology Machine Learning 62F10, 62J12, 68T07 We present a methodology for using unlabeled data to design semi-supervised learning (SSL) methods that improve the predictive performance of supervised learning for regression tasks. The main idea is to design different mechanisms for integrating the unlabeled data, and include in each of them a mixing parameter $α$, controlling the weight given to the unlabeled data. Focusing on Generalized Linear Models (GLM) and linear interpolators classes of models, we analyze the characteristics of different mixing mechanisms, and prove that it is consistently beneficial to integrate the unlabeled data with some nonzero mixing ratio $α>0$, in terms of predictive performance. Moreover, we provide a rigorous framework to estimate the best mixing ratio where mixed-SSL delivers the best predictive performance, while using the labeled and unlabeled data on hand. The effectiveness of our methodology in delivering substantial improvement compared to the standard supervised models, in a variety of settings, is demonstrated empirically through extensive simulation, providing empirical support for our theoretical analysis. We also demonstrate the applicability of our methodology (with some heuristic modifications) to improve more complex models, such as deep neural networks, in real-world regression tasks |
| title | Mixed Semi-Supervised Generalized-Linear-Regression with Applications to Deep-Learning and Interpolators |
| topic | Methodology Machine Learning 62F10, 62J12, 68T07 |
| url | https://arxiv.org/abs/2302.09526 |