The unbearable lightness of Restricted Boltzmann Machines: Theoretical Insights and Biological Applications
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910777032572928 |
|---|---|
| author | di Sarra, Giovanni Bravi, Barbara Roudi, Yasser |
| author_facet | di Sarra, Giovanni Bravi, Barbara Roudi, Yasser |
| contents | Restricted Boltzmann Machines are simple yet powerful neural networks. They can be used for learning structure in data, and are used as a building block of more complex neural architectures. At the same time, their simplicity makes them easy to use, amenable to theoretical analysis, yielding interpretable models in applications. Here, we focus on reviewing the role that the activation functions, describing the input-output relationship of single neurons in RBM, play in the functionality of these models. We discuss recent theoretical results on the benefits and limitations of different activation functions. We also review applications to biological data analysis, namely neural data analysis, where RBM units are mostly taken to have sigmoid activation functions and binary units, to protein data analysis and immunology where non-binary units and non-sigmoid activation functions have recently been shown to yield important insights into the data. Finally, we discuss open problems addressing which can shed light on broader issues in neural network research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_04387 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The unbearable lightness of Restricted Boltzmann Machines: Theoretical Insights and Biological Applications di Sarra, Giovanni Bravi, Barbara Roudi, Yasser Disordered Systems and Neural Networks Machine Learning Data Analysis, Statistics and Probability Restricted Boltzmann Machines are simple yet powerful neural networks. They can be used for learning structure in data, and are used as a building block of more complex neural architectures. At the same time, their simplicity makes them easy to use, amenable to theoretical analysis, yielding interpretable models in applications. Here, we focus on reviewing the role that the activation functions, describing the input-output relationship of single neurons in RBM, play in the functionality of these models. We discuss recent theoretical results on the benefits and limitations of different activation functions. We also review applications to biological data analysis, namely neural data analysis, where RBM units are mostly taken to have sigmoid activation functions and binary units, to protein data analysis and immunology where non-binary units and non-sigmoid activation functions have recently been shown to yield important insights into the data. Finally, we discuss open problems addressing which can shed light on broader issues in neural network research. |
| title | The unbearable lightness of Restricted Boltzmann Machines: Theoretical Insights and Biological Applications |
| topic | Disordered Systems and Neural Networks Machine Learning Data Analysis, Statistics and Probability |
| url | https://arxiv.org/abs/2501.04387 |