Neuromodulation via Krotov-Hopfield Improves Accuracy and Robustness of RBMs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917188722491392 |
|---|---|
| author | Tambaş, Başer Subaşı, A. Levent Kabakçıoğlu, Alkan |
| author_facet | Tambaş, Başer Subaşı, A. Levent Kabakçıoğlu, Alkan |
| contents | In biological systems, neuromodulation tunes synaptic plasticity based on the internal state of the organism, complementing stimulus-driven Hebbian learning. The algorithm recently proposed by Krotov and Hopfield \cite{krotov_2019} can be utilized to mirror this process in artificial neural networks, where its built-in intra-layer competition and selective inhibition of synaptic updates offer a cost-effective remedy for the lack of lateral connections through a simplified attention mechanism. We demonstrate that KH-modulated RBMs outperform standard (shallow) RBMs in both reconstruction and classification tasks, offering a superior trade-off between generalization performance and model size, with the additional benefit of robustness to weight initialization as well as to overfitting during training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_06902 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Neuromodulation via Krotov-Hopfield Improves Accuracy and Robustness of RBMs Tambaş, Başer Subaşı, A. Levent Kabakçıoğlu, Alkan Disordered Systems and Neural Networks In biological systems, neuromodulation tunes synaptic plasticity based on the internal state of the organism, complementing stimulus-driven Hebbian learning. The algorithm recently proposed by Krotov and Hopfield \cite{krotov_2019} can be utilized to mirror this process in artificial neural networks, where its built-in intra-layer competition and selective inhibition of synaptic updates offer a cost-effective remedy for the lack of lateral connections through a simplified attention mechanism. We demonstrate that KH-modulated RBMs outperform standard (shallow) RBMs in both reconstruction and classification tasks, offering a superior trade-off between generalization performance and model size, with the additional benefit of robustness to weight initialization as well as to overfitting during training. |
| title | Neuromodulation via Krotov-Hopfield Improves Accuracy and Robustness of RBMs |
| topic | Disordered Systems and Neural Networks |
| url | https://arxiv.org/abs/2505.06902 |