Towards Efficient and Reliable AI Through Neuromorphic Principles
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911264736804864 |
|---|---|
| author | Rajendran, Bipin Simeone, Osvaldo Al-Hashimi, Bashir M. |
| author_facet | Rajendran, Bipin Simeone, Osvaldo Al-Hashimi, Bashir M. |
| contents | Artificial intelligence (AI) research today is largely driven by ever-larger neural network models trained on graphics processing units (GPUs). This paradigm has yielded remarkable progress, but it also risks entrenching a hardware lottery in which algorithmic choices succeed primarily because they align with current hardware, rather than because they are inherently superior. In particular, the dominance of Transformer architectures running on GPU clusters has led to an arms race of scaling up models, resulting in exorbitant computational costs and energy usage. At the same time, today's AI models often remain unreliable in the sense that they cannot properly quantify uncertainty in their decisions -- for example, large language models tend to hallucinate incorrect outputs with high confidence.
This article argues that achieving more efficient and reliable AI will require embracing a set of principles that are well-aligned with the goals of neuromorphic engineering, which are in turn inspired by how the brain processes information. Specifically, we outline six key neuromorphic principles, spanning algorithms, architectures, and hardware, that can inform the design of future AI systems: (i) the use of stateful, recurrent models; (ii) extreme dynamic sparsity, possibly down to spike-based processing; (iii) backpropagation-free on-device learning and fine-tuning; (iv) probabilistic decision-making; (v) in-memory computing; and (vi) hardware-software co-design via stochastic computing. We discuss each of these principles in turn, surveying relevant prior work and pointing to directions for research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_15942 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Towards Efficient and Reliable AI Through Neuromorphic Principles Rajendran, Bipin Simeone, Osvaldo Al-Hashimi, Bashir M. Artificial Intelligence Emerging Technologies Information Theory Artificial intelligence (AI) research today is largely driven by ever-larger neural network models trained on graphics processing units (GPUs). This paradigm has yielded remarkable progress, but it also risks entrenching a hardware lottery in which algorithmic choices succeed primarily because they align with current hardware, rather than because they are inherently superior. In particular, the dominance of Transformer architectures running on GPU clusters has led to an arms race of scaling up models, resulting in exorbitant computational costs and energy usage. At the same time, today's AI models often remain unreliable in the sense that they cannot properly quantify uncertainty in their decisions -- for example, large language models tend to hallucinate incorrect outputs with high confidence. This article argues that achieving more efficient and reliable AI will require embracing a set of principles that are well-aligned with the goals of neuromorphic engineering, which are in turn inspired by how the brain processes information. Specifically, we outline six key neuromorphic principles, spanning algorithms, architectures, and hardware, that can inform the design of future AI systems: (i) the use of stateful, recurrent models; (ii) extreme dynamic sparsity, possibly down to spike-based processing; (iii) backpropagation-free on-device learning and fine-tuning; (iv) probabilistic decision-making; (v) in-memory computing; and (vi) hardware-software co-design via stochastic computing. We discuss each of these principles in turn, surveying relevant prior work and pointing to directions for research. |
| title | Towards Efficient and Reliable AI Through Neuromorphic Principles |
| topic | Artificial Intelligence Emerging Technologies Information Theory |
| url | https://arxiv.org/abs/2309.15942 |