Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2404.06972 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913308568715264 |
|---|---|
| author | Abbas, Konaté Mohamed Yao, Anne-Françoise Chateau, Thierry Bouges, Pierre |
| author_facet | Abbas, Konaté Mohamed Yao, Anne-Françoise Chateau, Thierry Bouges, Pierre |
| contents | In this paper, we investigate continual learning performance metrics used in class incremental learning strategies for continual learning (CL) using some high performing methods. We investigate especially mean task accuracy. First, we show that it lacks of expressiveness through some simple experiments to capture performance. We show that monitoring average tasks performance is over optimistic and can lead to misleading conclusions for future real life industrial uses. Then, we propose first a simple metric, Minimal Incremental Class Accuracy (MICA) which gives a fair and more useful evaluation of different continual learning methods. Moreover, in order to provide a simple way to easily compare different methods performance in continual learning, we derive another single scalar metric that take into account the learning performance variation as well as our newly introduced metric. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_06972 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Toward industrial use of continual learning : new metrics proposal for class incremental learning Abbas, Konaté Mohamed Yao, Anne-Françoise Chateau, Thierry Bouges, Pierre Machine Learning Artificial Intelligence In this paper, we investigate continual learning performance metrics used in class incremental learning strategies for continual learning (CL) using some high performing methods. We investigate especially mean task accuracy. First, we show that it lacks of expressiveness through some simple experiments to capture performance. We show that monitoring average tasks performance is over optimistic and can lead to misleading conclusions for future real life industrial uses. Then, we propose first a simple metric, Minimal Incremental Class Accuracy (MICA) which gives a fair and more useful evaluation of different continual learning methods. Moreover, in order to provide a simple way to easily compare different methods performance in continual learning, we derive another single scalar metric that take into account the learning performance variation as well as our newly introduced metric. |
| title | Toward industrial use of continual learning : new metrics proposal for class incremental learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2404.06972 |