Robust Semi-supervised Learning via $f$-Divergence and $α$-Rényi Divergence
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913338237124608 |
|---|---|
| author | Aminian, Gholamali Bagheri, Amirhossien JafariNodeh, Mahyar Karimian, Radmehr Yassaee, Mohammad-Hossein |
| author_facet | Aminian, Gholamali Bagheri, Amirhossien JafariNodeh, Mahyar Karimian, Radmehr Yassaee, Mohammad-Hossein |
| contents | This paper investigates a range of empirical risk functions and regularization methods suitable for self-training methods in semi-supervised learning. These approaches draw inspiration from various divergence measures, such as $f$-divergences and $α$-Rényi divergences. Inspired by the theoretical foundations rooted in divergences, i.e., $f$-divergences and $α$-Rényi divergence, we also provide valuable insights to enhance the understanding of our empirical risk functions and regularization techniques. In the pseudo-labeling and entropy minimization techniques as self-training methods for effective semi-supervised learning, the self-training process has some inherent mismatch between the true label and pseudo-label (noisy pseudo-labels) and some of our empirical risk functions are robust, concerning noisy pseudo-labels. Under some conditions, our empirical risk functions demonstrate better performance when compared to traditional self-training methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_00454 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Robust Semi-supervised Learning via $f$-Divergence and $α$-Rényi Divergence Aminian, Gholamali Bagheri, Amirhossien JafariNodeh, Mahyar Karimian, Radmehr Yassaee, Mohammad-Hossein Machine Learning Information Theory This paper investigates a range of empirical risk functions and regularization methods suitable for self-training methods in semi-supervised learning. These approaches draw inspiration from various divergence measures, such as $f$-divergences and $α$-Rényi divergences. Inspired by the theoretical foundations rooted in divergences, i.e., $f$-divergences and $α$-Rényi divergence, we also provide valuable insights to enhance the understanding of our empirical risk functions and regularization techniques. In the pseudo-labeling and entropy minimization techniques as self-training methods for effective semi-supervised learning, the self-training process has some inherent mismatch between the true label and pseudo-label (noisy pseudo-labels) and some of our empirical risk functions are robust, concerning noisy pseudo-labels. Under some conditions, our empirical risk functions demonstrate better performance when compared to traditional self-training methods. |
| title | Robust Semi-supervised Learning via $f$-Divergence and $α$-Rényi Divergence |
| topic | Machine Learning Information Theory |
| url | https://arxiv.org/abs/2405.00454 |