Sleep-Like Unsupervised Replay Improves Performance when Data are Limited or Unbalanced
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929246400675840 |
|---|---|
| author | Bazhenov, Anthony Dewasurendra, Pahan Krishnan, Giri Delanois, Jean Erik |
| author_facet | Bazhenov, Anthony Dewasurendra, Pahan Krishnan, Giri Delanois, Jean Erik |
| contents | The performance of artificial neural networks (ANNs) degrades when training data are limited or imbalanced. In contrast, the human brain can learn quickly from just a few examples. Here, we investigated the role of sleep in improving the performance of ANNs trained with limited data on the MNIST and Fashion MNIST datasets. Sleep was implemented as an unsupervised phase with local Hebbian type learning rules. We found a significant boost in accuracy after the sleep phase for models trained with limited data in the range of 0.5-10% of total MNIST or Fashion MNIST datasets. When more than 10% of the total data was used, sleep alone had a slight negative impact on performance, but this was remedied by fine-tuning on the original data. This study sheds light on a potential synaptic weight dynamics strategy employed by the brain during sleep to enhance memory performance when training data are limited or imbalanced. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_10956 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Sleep-Like Unsupervised Replay Improves Performance when Data are Limited or Unbalanced Bazhenov, Anthony Dewasurendra, Pahan Krishnan, Giri Delanois, Jean Erik Neural and Evolutionary Computing Machine Learning The performance of artificial neural networks (ANNs) degrades when training data are limited or imbalanced. In contrast, the human brain can learn quickly from just a few examples. Here, we investigated the role of sleep in improving the performance of ANNs trained with limited data on the MNIST and Fashion MNIST datasets. Sleep was implemented as an unsupervised phase with local Hebbian type learning rules. We found a significant boost in accuracy after the sleep phase for models trained with limited data in the range of 0.5-10% of total MNIST or Fashion MNIST datasets. When more than 10% of the total data was used, sleep alone had a slight negative impact on performance, but this was remedied by fine-tuning on the original data. This study sheds light on a potential synaptic weight dynamics strategy employed by the brain during sleep to enhance memory performance when training data are limited or imbalanced. |
| title | Sleep-Like Unsupervised Replay Improves Performance when Data are Limited or Unbalanced |
| topic | Neural and Evolutionary Computing Machine Learning |
| url | https://arxiv.org/abs/2402.10956 |