Spike-timing-dependent Hebbian learning as noisy gradient descent
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918285020233728 |
|---|---|
| author | Dexheimer, Niklas Gaudlitz, Sascha Schmidt-Hieber, Johannes |
| author_facet | Dexheimer, Niklas Gaudlitz, Sascha Schmidt-Hieber, Johannes |
| contents | Hebbian learning is a key principle underlying learning in biological neural networks. We relate a Hebbian spike-timing-dependent plasticity rule to noisy gradient descent with respect to a non-convex loss function on the probability simplex. Despite the constant injection of noise and the non-convexity of the underlying optimization problem, one can rigorously prove that the considered Hebbian learning dynamic identifies the presynaptic neuron with the highest activity and that the convergence is exponentially fast in the number of iterations. This is non-standard and surprising as typically noisy gradient descent with fixed noise level only converges to a stationary regime where the noise causes the dynamic to fluctuate around a minimiser. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_10272 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Spike-timing-dependent Hebbian learning as noisy gradient descent Dexheimer, Niklas Gaudlitz, Sascha Schmidt-Hieber, Johannes Machine Learning Statistics Theory Hebbian learning is a key principle underlying learning in biological neural networks. We relate a Hebbian spike-timing-dependent plasticity rule to noisy gradient descent with respect to a non-convex loss function on the probability simplex. Despite the constant injection of noise and the non-convexity of the underlying optimization problem, one can rigorously prove that the considered Hebbian learning dynamic identifies the presynaptic neuron with the highest activity and that the convergence is exponentially fast in the number of iterations. This is non-standard and surprising as typically noisy gradient descent with fixed noise level only converges to a stationary regime where the noise causes the dynamic to fluctuate around a minimiser. |
| title | Spike-timing-dependent Hebbian learning as noisy gradient descent |
| topic | Machine Learning Statistics Theory |
| url | https://arxiv.org/abs/2505.10272 |