Training of Physical Neural Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909217601880064 |
|---|---|
| author | Momeni, Ali Rahmani, Babak Scellier, Benjamin Wright, Logan G. McMahon, Peter L. Wanjura, Clara C. Li, Yuhang Skalli, Anas Berloff, Natalia G. Onodera, Tatsuhiro Oguz, Ilker Morichetti, Francesco del Hougne, Philipp Gallo, Manuel Le Sebastian, Abu Mirhoseini, Azalia Zhang, Cheng Marković, Danijela Brunner, Daniel Moser, Christophe Gigan, Sylvain Marquardt, Florian Ozcan, Aydogan Grollier, Julie Liu, Andrea J. Psaltis, Demetri Alù, Andrea Fleury, Romain |
| author_facet | Momeni, Ali Rahmani, Babak Scellier, Benjamin Wright, Logan G. McMahon, Peter L. Wanjura, Clara C. Li, Yuhang Skalli, Anas Berloff, Natalia G. Onodera, Tatsuhiro Oguz, Ilker Morichetti, Francesco del Hougne, Philipp Gallo, Manuel Le Sebastian, Abu Mirhoseini, Azalia Zhang, Cheng Marković, Danijela Brunner, Daniel Moser, Christophe Gigan, Sylvain Marquardt, Florian Ozcan, Aydogan Grollier, Julie Liu, Andrea J. Psaltis, Demetri Alù, Andrea Fleury, Romain |
| contents | Physical neural networks (PNNs) are a class of neural-like networks that leverage the properties of physical systems to perform computation. While PNNs are so far a niche research area with small-scale laboratory demonstrations, they are arguably one of the most underappreciated important opportunities in modern AI. Could we train AI models 1000x larger than current ones? Could we do this and also have them perform inference locally and privately on edge devices, such as smartphones or sensors? Research over the past few years has shown that the answer to all these questions is likely "yes, with enough research": PNNs could one day radically change what is possible and practical for AI systems. To do this will however require rethinking both how AI models work, and how they are trained - primarily by considering the problems through the constraints of the underlying hardware physics. To train PNNs at large scale, many methods including backpropagation-based and backpropagation-free approaches are now being explored. These methods have various trade-offs, and so far no method has been shown to scale to the same scale and performance as the backpropagation algorithm widely used in deep learning today. However, this is rapidly changing, and a diverse ecosystem of training techniques provides clues for how PNNs may one day be utilized to create both more efficient realizations of current-scale AI models, and to enable unprecedented-scale models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_03372 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Training of Physical Neural Networks Momeni, Ali Rahmani, Babak Scellier, Benjamin Wright, Logan G. McMahon, Peter L. Wanjura, Clara C. Li, Yuhang Skalli, Anas Berloff, Natalia G. Onodera, Tatsuhiro Oguz, Ilker Morichetti, Francesco del Hougne, Philipp Gallo, Manuel Le Sebastian, Abu Mirhoseini, Azalia Zhang, Cheng Marković, Danijela Brunner, Daniel Moser, Christophe Gigan, Sylvain Marquardt, Florian Ozcan, Aydogan Grollier, Julie Liu, Andrea J. Psaltis, Demetri Alù, Andrea Fleury, Romain Applied Physics Machine Learning Physical neural networks (PNNs) are a class of neural-like networks that leverage the properties of physical systems to perform computation. While PNNs are so far a niche research area with small-scale laboratory demonstrations, they are arguably one of the most underappreciated important opportunities in modern AI. Could we train AI models 1000x larger than current ones? Could we do this and also have them perform inference locally and privately on edge devices, such as smartphones or sensors? Research over the past few years has shown that the answer to all these questions is likely "yes, with enough research": PNNs could one day radically change what is possible and practical for AI systems. To do this will however require rethinking both how AI models work, and how they are trained - primarily by considering the problems through the constraints of the underlying hardware physics. To train PNNs at large scale, many methods including backpropagation-based and backpropagation-free approaches are now being explored. These methods have various trade-offs, and so far no method has been shown to scale to the same scale and performance as the backpropagation algorithm widely used in deep learning today. However, this is rapidly changing, and a diverse ecosystem of training techniques provides clues for how PNNs may one day be utilized to create both more efficient realizations of current-scale AI models, and to enable unprecedented-scale models. |
| title | Training of Physical Neural Networks |
| topic | Applied Physics Machine Learning |
| url | https://arxiv.org/abs/2406.03372 |