Energy--Information Trade-off Induces Continuous and Discontinuous Phase Transitions in Lateral Predictive Coding
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929385304489984 |
|---|---|
| author | Huang, Zhen-Ye Zhou, Ruyi Huang, Miao Zhou, Hai-Jun |
| author_facet | Huang, Zhen-Ye Zhou, Ruyi Huang, Miao Zhou, Hai-Jun |
| contents | Lateral predictive coding is a recurrent neural network which creates energy-efficient internal representations by exploiting statistical regularity in sensory inputs. Here we investigate the trade-off between information robustness and energy in a linear model of lateral predictive coding analytically and by numerical minimization of a free energy. We observe several phase transitions in the synaptic weight matrix, especially a continuous transition which breaks reciprocity and permutation symmetry and builds cyclic dominance and a discontinuous transition with the associated sudden emergence of tight balance between excitatory and inhibitory interactions. The optimal network follows an ideal-gas law in an extended temperature range and saturates the efficiency upper-bound of energy utilization. These results bring theoretical insights on the emergence and evolution of complex internal models in predictive processing systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_11681 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Energy--Information Trade-off Induces Continuous and Discontinuous Phase Transitions in Lateral Predictive Coding Huang, Zhen-Ye Zhou, Ruyi Huang, Miao Zhou, Hai-Jun Neurons and Cognition Disordered Systems and Neural Networks Lateral predictive coding is a recurrent neural network which creates energy-efficient internal representations by exploiting statistical regularity in sensory inputs. Here we investigate the trade-off between information robustness and energy in a linear model of lateral predictive coding analytically and by numerical minimization of a free energy. We observe several phase transitions in the synaptic weight matrix, especially a continuous transition which breaks reciprocity and permutation symmetry and builds cyclic dominance and a discontinuous transition with the associated sudden emergence of tight balance between excitatory and inhibitory interactions. The optimal network follows an ideal-gas law in an extended temperature range and saturates the efficiency upper-bound of energy utilization. These results bring theoretical insights on the emergence and evolution of complex internal models in predictive processing systems. |
| title | Energy--Information Trade-off Induces Continuous and Discontinuous Phase Transitions in Lateral Predictive Coding |
| topic | Neurons and Cognition Disordered Systems and Neural Networks |
| url | https://arxiv.org/abs/2302.11681 |