Enhancing Reliability of Neural Networks at the Edge: Inverted Normalization with Stochastic Affine Transformations
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917572665933824 |
|---|---|
| author | Ahmed, Soyed Tuhin Danouchi, Kamal Prenat, Guillaume Anghel, Lorena Tahoori, Mehdi B. |
| author_facet | Ahmed, Soyed Tuhin Danouchi, Kamal Prenat, Guillaume Anghel, Lorena Tahoori, Mehdi B. |
| contents | Bayesian Neural Networks (BayNNs) naturally provide uncertainty in their predictions, making them a suitable choice in safety-critical applications. Additionally, their realization using memristor-based in-memory computing (IMC) architectures enables them for resource-constrained edge applications. In addition to predictive uncertainty, however, the ability to be inherently robust to noise in computation is also essential to ensure functional safety. In particular, memristor-based IMCs are susceptible to various sources of non-idealities such as manufacturing and runtime variations, drift, and failure, which can significantly reduce inference accuracy. In this paper, we propose a method to inherently enhance the robustness and inference accuracy of BayNNs deployed in IMC architectures. To achieve this, we introduce a novel normalization layer combined with stochastic affine transformations. Empirical results in various benchmark datasets show a graceful degradation in inference accuracy, with an improvement of up to $58.11\%$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_12416 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Enhancing Reliability of Neural Networks at the Edge: Inverted Normalization with Stochastic Affine Transformations Ahmed, Soyed Tuhin Danouchi, Kamal Prenat, Guillaume Anghel, Lorena Tahoori, Mehdi B. Machine Learning Hardware Architecture Emerging Technologies Bayesian Neural Networks (BayNNs) naturally provide uncertainty in their predictions, making them a suitable choice in safety-critical applications. Additionally, their realization using memristor-based in-memory computing (IMC) architectures enables them for resource-constrained edge applications. In addition to predictive uncertainty, however, the ability to be inherently robust to noise in computation is also essential to ensure functional safety. In particular, memristor-based IMCs are susceptible to various sources of non-idealities such as manufacturing and runtime variations, drift, and failure, which can significantly reduce inference accuracy. In this paper, we propose a method to inherently enhance the robustness and inference accuracy of BayNNs deployed in IMC architectures. To achieve this, we introduce a novel normalization layer combined with stochastic affine transformations. Empirical results in various benchmark datasets show a graceful degradation in inference accuracy, with an improvement of up to $58.11\%$. |
| title | Enhancing Reliability of Neural Networks at the Edge: Inverted Normalization with Stochastic Affine Transformations |
| topic | Machine Learning Hardware Architecture Emerging Technologies |
| url | https://arxiv.org/abs/2401.12416 |