Learning efficient backprojections across cortical hierarchies in real time
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2022
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913219974529024 |
|---|---|
| author | Max, Kevin Kriener, Laura García, Garibaldi Pineda Nowotny, Thomas Jaras, Ismael Senn, Walter Petrovici, Mihai A. |
| author_facet | Max, Kevin Kriener, Laura García, Garibaldi Pineda Nowotny, Thomas Jaras, Ismael Senn, Walter Petrovici, Mihai A. |
| contents | Models of sensory processing and learning in the cortex need to efficiently assign credit to synapses in all areas. In deep learning, a known solution is error backpropagation, which however requires biologically implausible weight transport from feed-forward to feedback paths.
We introduce Phaseless Alignment Learning (PAL), a bio-plausible method to learn efficient feedback weights in layered cortical hierarchies. This is achieved by exploiting the noise naturally found in biophysical systems as an additional carrier of information. In our dynamical system, all weights are learned simultaneously with always-on plasticity and using only information locally available to the synapses. Our method is completely phase-free (no forward and backward passes or phased learning) and allows for efficient error propagation across multi-layer cortical hierarchies, while maintaining biologically plausible signal transport and learning.
Our method is applicable to a wide class of models and improves on previously known biologically plausible ways of credit assignment: compared to random synaptic feedback, it can solve complex tasks with less neurons and learn more useful latent representations. We demonstrate this on various classification tasks using a cortical microcircuit model with prospective coding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2212_10249 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Learning efficient backprojections across cortical hierarchies in real time Max, Kevin Kriener, Laura García, Garibaldi Pineda Nowotny, Thomas Jaras, Ismael Senn, Walter Petrovici, Mihai A. Neurons and Cognition Machine Learning Neural and Evolutionary Computing Models of sensory processing and learning in the cortex need to efficiently assign credit to synapses in all areas. In deep learning, a known solution is error backpropagation, which however requires biologically implausible weight transport from feed-forward to feedback paths. We introduce Phaseless Alignment Learning (PAL), a bio-plausible method to learn efficient feedback weights in layered cortical hierarchies. This is achieved by exploiting the noise naturally found in biophysical systems as an additional carrier of information. In our dynamical system, all weights are learned simultaneously with always-on plasticity and using only information locally available to the synapses. Our method is completely phase-free (no forward and backward passes or phased learning) and allows for efficient error propagation across multi-layer cortical hierarchies, while maintaining biologically plausible signal transport and learning. Our method is applicable to a wide class of models and improves on previously known biologically plausible ways of credit assignment: compared to random synaptic feedback, it can solve complex tasks with less neurons and learn more useful latent representations. We demonstrate this on various classification tasks using a cortical microcircuit model with prospective coding. |
| title | Learning efficient backprojections across cortical hierarchies in real time |
| topic | Neurons and Cognition Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2212.10249 |