Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.06902 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917188722491392 |
|---|---|
| author | Tambaş, Başer Subaşı, A. Levent Kabakçıoğlu, Alkan |
| author_facet | Tambaş, Başer Subaşı, A. Levent Kabakçıoğlu, Alkan |
| contents | In biological systems, neuromodulation tunes synaptic plasticity based on the internal state of the organism, complementing stimulus-driven Hebbian learning. The algorithm recently proposed by Krotov and Hopfield \cite{krotov_2019} can be utilized to mirror this process in artificial neural networks, where its built-in intra-layer competition and selective inhibition of synaptic updates offer a cost-effective remedy for the lack of lateral connections through a simplified attention mechanism. We demonstrate that KH-modulated RBMs outperform standard (shallow) RBMs in both reconstruction and classification tasks, offering a superior trade-off between generalization performance and model size, with the additional benefit of robustness to weight initialization as well as to overfitting during training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_06902 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Neuromodulation via Krotov-Hopfield Improves Accuracy and Robustness of RBMs Tambaş, Başer Subaşı, A. Levent Kabakçıoğlu, Alkan Disordered Systems and Neural Networks In biological systems, neuromodulation tunes synaptic plasticity based on the internal state of the organism, complementing stimulus-driven Hebbian learning. The algorithm recently proposed by Krotov and Hopfield \cite{krotov_2019} can be utilized to mirror this process in artificial neural networks, where its built-in intra-layer competition and selective inhibition of synaptic updates offer a cost-effective remedy for the lack of lateral connections through a simplified attention mechanism. We demonstrate that KH-modulated RBMs outperform standard (shallow) RBMs in both reconstruction and classification tasks, offering a superior trade-off between generalization performance and model size, with the additional benefit of robustness to weight initialization as well as to overfitting during training. |
| title | Neuromodulation via Krotov-Hopfield Improves Accuracy and Robustness of RBMs |
| topic | Disordered Systems and Neural Networks |
| url | https://arxiv.org/abs/2505.06902 |