Analog Bayesian neural networks are insensitive to the shape of the weight distribution
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910778777403392 |
|---|---|
| author | Patel, Ravi G. Xiao, T. Patrick Agarwal, Sapan Bennett, Christopher |
| author_facet | Patel, Ravi G. Xiao, T. Patrick Agarwal, Sapan Bennett, Christopher |
| contents | Recent work has demonstrated that Bayesian neural networks (BNN's) trained with mean field variational inference (MFVI) can be implemented in analog hardware, promising orders of magnitude energy savings compared to the standard digital implementations. However, while Gaussians are typically used as the variational distribution in MFVI, it is difficult to precisely control the shape of the noise distributions produced by sampling analog devices. This paper introduces a method for MFVI training using real device noise as the variational distribution. Furthermore, we demonstrate empirically that the predictive distributions from BNN's with the same weight means and variances converge to the same distribution, regardless of the shape of the variational distribution. This result suggests that analog device designers do not need to consider the shape of the device noise distribution when hardware-implementing BNNs performing MFVI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_05564 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Analog Bayesian neural networks are insensitive to the shape of the weight distribution Patel, Ravi G. Xiao, T. Patrick Agarwal, Sapan Bennett, Christopher Machine Learning Hardware Architecture Recent work has demonstrated that Bayesian neural networks (BNN's) trained with mean field variational inference (MFVI) can be implemented in analog hardware, promising orders of magnitude energy savings compared to the standard digital implementations. However, while Gaussians are typically used as the variational distribution in MFVI, it is difficult to precisely control the shape of the noise distributions produced by sampling analog devices. This paper introduces a method for MFVI training using real device noise as the variational distribution. Furthermore, we demonstrate empirically that the predictive distributions from BNN's with the same weight means and variances converge to the same distribution, regardless of the shape of the variational distribution. This result suggests that analog device designers do not need to consider the shape of the device noise distribution when hardware-implementing BNNs performing MFVI. |
| title | Analog Bayesian neural networks are insensitive to the shape of the weight distribution |
| topic | Machine Learning Hardware Architecture |
| url | https://arxiv.org/abs/2501.05564 |