Improving deep neural network performance through sampling
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918468591288320 |
|---|---|
| author | Ghantasala, Lakshmi A. Li, Ming-Che Jaiswal, Risi Behin-Aein, Behtash Makin, Joseph Sen, Shreyas Datta, Supriyo |
| author_facet | Ghantasala, Lakshmi A. Li, Ming-Che Jaiswal, Risi Behin-Aein, Behtash Makin, Joseph Sen, Shreyas Datta, Supriyo |
| contents | Energy efficient sampling with probabilistic neurons or p-bits has been demonstrated in the context of Boltzmann machines and it is natural to ask if these approaches can be extended to the field of generative AI where energy costs have become prohibitively large. However, this very active field is dominated by feedforward deep neural networks (DNNs) which primarily use multi-bit deterministic neurons with no role for sampling. In this paper we first show that it is feasible to obtain superior accuracy through the use of multiple samples generated by probabilistic networks. This possibility raises the question of which option is energetically preferable for improving accuracy: generating more samples, or adding more bits to a single deterministic sample. We provide a simple expression that can be used to estimate these energy tradeoffs and illustrate it with results for different algorithms and architectures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_07763 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Improving deep neural network performance through sampling Ghantasala, Lakshmi A. Li, Ming-Che Jaiswal, Risi Behin-Aein, Behtash Makin, Joseph Sen, Shreyas Datta, Supriyo Disordered Systems and Neural Networks Energy efficient sampling with probabilistic neurons or p-bits has been demonstrated in the context of Boltzmann machines and it is natural to ask if these approaches can be extended to the field of generative AI where energy costs have become prohibitively large. However, this very active field is dominated by feedforward deep neural networks (DNNs) which primarily use multi-bit deterministic neurons with no role for sampling. In this paper we first show that it is feasible to obtain superior accuracy through the use of multiple samples generated by probabilistic networks. This possibility raises the question of which option is energetically preferable for improving accuracy: generating more samples, or adding more bits to a single deterministic sample. We provide a simple expression that can be used to estimate these energy tradeoffs and illustrate it with results for different algorithms and architectures. |
| title | Improving deep neural network performance through sampling |
| topic | Disordered Systems and Neural Networks |
| url | https://arxiv.org/abs/2507.07763 |