Towards Efficient Neuro-Symbolic AI: From Workload Characterization to Hardware Architecture
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929509528240128 |
|---|---|
| author | Wan, Zishen Liu, Che-Kai Yang, Hanchen Raj, Ritik Li, Chaojian You, Haoran Fu, Yonggan Wan, Cheng Li, Sixu Kim, Youbin Samajdar, Ananda Lin, Yingyan Celine Ibrahim, Mohamed Rabaey, Jan M. Krishna, Tushar Raychowdhury, Arijit |
| author_facet | Wan, Zishen Liu, Che-Kai Yang, Hanchen Raj, Ritik Li, Chaojian You, Haoran Fu, Yonggan Wan, Cheng Li, Sixu Kim, Youbin Samajdar, Ananda Lin, Yingyan Celine Ibrahim, Mohamed Rabaey, Jan M. Krishna, Tushar Raychowdhury, Arijit |
| contents | The remarkable advancements in artificial intelligence (AI), primarily driven by deep neural networks, are facing challenges surrounding unsustainable computational trajectories, limited robustness, and a lack of explainability. To develop next-generation cognitive AI systems, neuro-symbolic AI emerges as a promising paradigm, fusing neural and symbolic approaches to enhance interpretability, robustness, and trustworthiness, while facilitating learning from much less data. Recent neuro-symbolic systems have demonstrated great potential in collaborative human-AI scenarios with reasoning and cognitive capabilities. In this paper, we aim to understand the workload characteristics and potential architectures for neuro-symbolic AI. We first systematically categorize neuro-symbolic AI algorithms, and then experimentally evaluate and analyze them in terms of runtime, memory, computational operators, sparsity, and system characteristics on CPUs, GPUs, and edge SoCs. Our studies reveal that neuro-symbolic models suffer from inefficiencies on off-the-shelf hardware, due to the memory-bound nature of vector-symbolic and logical operations, complex flow control, data dependencies, sparsity variations, and limited scalability. Based on profiling insights, we suggest cross-layer optimization solutions and present a hardware acceleration case study for vector-symbolic architecture to improve the performance, efficiency, and scalability of neuro-symbolic computing. Finally, we discuss the challenges and potential future directions of neuro-symbolic AI from both system and architectural perspectives. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_13153 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Efficient Neuro-Symbolic AI: From Workload Characterization to Hardware Architecture Wan, Zishen Liu, Che-Kai Yang, Hanchen Raj, Ritik Li, Chaojian You, Haoran Fu, Yonggan Wan, Cheng Li, Sixu Kim, Youbin Samajdar, Ananda Lin, Yingyan Celine Ibrahim, Mohamed Rabaey, Jan M. Krishna, Tushar Raychowdhury, Arijit Hardware Architecture Artificial Intelligence The remarkable advancements in artificial intelligence (AI), primarily driven by deep neural networks, are facing challenges surrounding unsustainable computational trajectories, limited robustness, and a lack of explainability. To develop next-generation cognitive AI systems, neuro-symbolic AI emerges as a promising paradigm, fusing neural and symbolic approaches to enhance interpretability, robustness, and trustworthiness, while facilitating learning from much less data. Recent neuro-symbolic systems have demonstrated great potential in collaborative human-AI scenarios with reasoning and cognitive capabilities. In this paper, we aim to understand the workload characteristics and potential architectures for neuro-symbolic AI. We first systematically categorize neuro-symbolic AI algorithms, and then experimentally evaluate and analyze them in terms of runtime, memory, computational operators, sparsity, and system characteristics on CPUs, GPUs, and edge SoCs. Our studies reveal that neuro-symbolic models suffer from inefficiencies on off-the-shelf hardware, due to the memory-bound nature of vector-symbolic and logical operations, complex flow control, data dependencies, sparsity variations, and limited scalability. Based on profiling insights, we suggest cross-layer optimization solutions and present a hardware acceleration case study for vector-symbolic architecture to improve the performance, efficiency, and scalability of neuro-symbolic computing. Finally, we discuss the challenges and potential future directions of neuro-symbolic AI from both system and architectural perspectives. |
| title | Towards Efficient Neuro-Symbolic AI: From Workload Characterization to Hardware Architecture |
| topic | Hardware Architecture Artificial Intelligence |
| url | https://arxiv.org/abs/2409.13153 |