Microscopic imprints of learned solutions in adaptive resistor networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913627574894592 |
|---|---|
| author | Guzman, Marcel Martins, Felipe Stern, Menachem Liu, Andrea J. |
| author_facet | Guzman, Marcel Martins, Felipe Stern, Menachem Liu, Andrea J. |
| contents | In physical networks trained using supervised learning, physical parameters are adjusted to produce desired responses to inputs. An example is electrical contrastive local learning networks of nodes connected by edges that are resistors that adjust their conductances during training. When an edge conductance changes, it upsets the current balance of every node. In response, physics adjusts the node voltages to minimize the dissipated power. Learning in these systems is therefore a coupled double-optimization process, in which the network descends both a cost landscape in the high-dimensional space of edge conductances, and a physical landscape -- the power -- in the high-dimensional space of node voltages. Because of this coupling, the physical landscape of a trained network contains information about the learned task. Here we demonstrate that all the physical information relevant to the trained input-output relation can be captured by a susceptibility, an experimentally measurable quantity. We supplement our theoretical results with simulations to show that the susceptibility is positively correlated with functional importance and that we can extract physical insight into how the system performs the task from the conductances of highly susceptible edges. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_19356 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Microscopic imprints of learned solutions in adaptive resistor networks Guzman, Marcel Martins, Felipe Stern, Menachem Liu, Andrea J. Disordered Systems and Neural Networks Soft Condensed Matter Statistical Mechanics In physical networks trained using supervised learning, physical parameters are adjusted to produce desired responses to inputs. An example is electrical contrastive local learning networks of nodes connected by edges that are resistors that adjust their conductances during training. When an edge conductance changes, it upsets the current balance of every node. In response, physics adjusts the node voltages to minimize the dissipated power. Learning in these systems is therefore a coupled double-optimization process, in which the network descends both a cost landscape in the high-dimensional space of edge conductances, and a physical landscape -- the power -- in the high-dimensional space of node voltages. Because of this coupling, the physical landscape of a trained network contains information about the learned task. Here we demonstrate that all the physical information relevant to the trained input-output relation can be captured by a susceptibility, an experimentally measurable quantity. We supplement our theoretical results with simulations to show that the susceptibility is positively correlated with functional importance and that we can extract physical insight into how the system performs the task from the conductances of highly susceptible edges. |
| title | Microscopic imprints of learned solutions in adaptive resistor networks |
| topic | Disordered Systems and Neural Networks Soft Condensed Matter Statistical Mechanics |
| url | https://arxiv.org/abs/2412.19356 |