The Fisher-Rao Loss for Learning under Label Noise
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929342427168768 |
|---|---|
| author | Miyamoto, Henrique K. Meneghetti, Fábio C. C. Costa, Sueli I. R. |
| author_facet | Miyamoto, Henrique K. Meneghetti, Fábio C. C. Costa, Sueli I. R. |
| contents | Choosing a suitable loss function is essential when learning by empirical risk minimisation. In many practical cases, the datasets used for training a classifier may contain incorrect labels, which prompts the interest for using loss functions that are inherently robust to label noise. In this paper, we study the Fisher-Rao loss function, which emerges from the Fisher-Rao distance in the statistical manifold of discrete distributions. We derive an upper bound for the performance degradation in the presence of label noise, and analyse the learning speed of this loss. Comparing with other commonly used losses, we argue that the Fisher-Rao loss provides a natural trade-off between robustness and training dynamics. Numerical experiments with synthetic and MNIST datasets illustrate this performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2210_16401 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | The Fisher-Rao Loss for Learning under Label Noise Miyamoto, Henrique K. Meneghetti, Fábio C. C. Costa, Sueli I. R. Machine Learning Information Theory Choosing a suitable loss function is essential when learning by empirical risk minimisation. In many practical cases, the datasets used for training a classifier may contain incorrect labels, which prompts the interest for using loss functions that are inherently robust to label noise. In this paper, we study the Fisher-Rao loss function, which emerges from the Fisher-Rao distance in the statistical manifold of discrete distributions. We derive an upper bound for the performance degradation in the presence of label noise, and analyse the learning speed of this loss. Comparing with other commonly used losses, we argue that the Fisher-Rao loss provides a natural trade-off between robustness and training dynamics. Numerical experiments with synthetic and MNIST datasets illustrate this performance. |
| title | The Fisher-Rao Loss for Learning under Label Noise |
| topic | Machine Learning Information Theory |
| url | https://arxiv.org/abs/2210.16401 |