From Loop Nests to Silicon: Mapping AI Workloads onto AMD NPUs with MLIR-AIR
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912652894142464 |
|---|---|
| author | Wang, Erwei Bayliss, Samuel Bisca, Andra Blair, Zachary Chowdhary, Sangeeta Denolf, Kristof Fifield, Jeff Freiberger, Brandon Hunhoff, Erika James-Roxby, Phil Lo, Jack Melber, Joseph Neuendorffer, Stephen Richter, Eddie Rosti, Andre Setoain, Javier Singh, Gagandeep Taka, Endri Vasireddy, Pranathi Yu, Zhewen Zhang, Niansong Zhuang, Jinming |
| author_facet | Wang, Erwei Bayliss, Samuel Bisca, Andra Blair, Zachary Chowdhary, Sangeeta Denolf, Kristof Fifield, Jeff Freiberger, Brandon Hunhoff, Erika James-Roxby, Phil Lo, Jack Melber, Joseph Neuendorffer, Stephen Richter, Eddie Rosti, Andre Setoain, Javier Singh, Gagandeep Taka, Endri Vasireddy, Pranathi Yu, Zhewen Zhang, Niansong Zhuang, Jinming |
| contents | General-purpose compilers abstract away parallelism, locality, and synchronization, limiting their effectiveness on modern spatial architectures. As modern computing architectures increasingly rely on fine-grained control over data movement, execution order, and compute placement for performance, compiler infrastructure must provide explicit mechanisms for orchestrating compute and data to fully exploit such architectures. We introduce MLIR-AIR, a novel, open-source compiler stack built on MLIR that bridges the semantic gap between high-level workloads and fine-grained spatial architectures such as AMD's NPUs. MLIR-AIR defines the AIR dialect, which provides structured representations for asynchronous and hierarchical operations across compute and memory resources. AIR primitives allow the compiler to orchestrate spatial scheduling, distribute computation across hardware regions, and overlap communication with computation without relying on ad hoc runtime coordination or manual scheduling. We demonstrate MLIR-AIR's capabilities through two case studies: matrix multiplication and the multi-head attention block from the LLaMA 2 model. For matrix multiplication, MLIR-AIR achieves up to 78.7% compute efficiency and generates implementations with performance almost identical to state-of-the-art, hand-optimized matrix multiplication written using the lower-level, close-to-metal MLIR-AIE framework. For multi-head attention, we demonstrate that the AIR interface supports fused implementations using approximately 150 lines of code, enabling tractable expression of complex workloads with efficient mapping to spatial hardware. MLIR-AIR transforms high-level structured control flow into spatial programs that efficiently utilize the compute fabric and memory hierarchy of an NPU, leveraging asynchronous execution, tiling, and communication overlap through compiler-managed scheduling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_14871 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Loop Nests to Silicon: Mapping AI Workloads onto AMD NPUs with MLIR-AIR Wang, Erwei Bayliss, Samuel Bisca, Andra Blair, Zachary Chowdhary, Sangeeta Denolf, Kristof Fifield, Jeff Freiberger, Brandon Hunhoff, Erika James-Roxby, Phil Lo, Jack Melber, Joseph Neuendorffer, Stephen Richter, Eddie Rosti, Andre Setoain, Javier Singh, Gagandeep Taka, Endri Vasireddy, Pranathi Yu, Zhewen Zhang, Niansong Zhuang, Jinming Computation and Language Hardware Architecture Machine Learning General-purpose compilers abstract away parallelism, locality, and synchronization, limiting their effectiveness on modern spatial architectures. As modern computing architectures increasingly rely on fine-grained control over data movement, execution order, and compute placement for performance, compiler infrastructure must provide explicit mechanisms for orchestrating compute and data to fully exploit such architectures. We introduce MLIR-AIR, a novel, open-source compiler stack built on MLIR that bridges the semantic gap between high-level workloads and fine-grained spatial architectures such as AMD's NPUs. MLIR-AIR defines the AIR dialect, which provides structured representations for asynchronous and hierarchical operations across compute and memory resources. AIR primitives allow the compiler to orchestrate spatial scheduling, distribute computation across hardware regions, and overlap communication with computation without relying on ad hoc runtime coordination or manual scheduling. We demonstrate MLIR-AIR's capabilities through two case studies: matrix multiplication and the multi-head attention block from the LLaMA 2 model. For matrix multiplication, MLIR-AIR achieves up to 78.7% compute efficiency and generates implementations with performance almost identical to state-of-the-art, hand-optimized matrix multiplication written using the lower-level, close-to-metal MLIR-AIE framework. For multi-head attention, we demonstrate that the AIR interface supports fused implementations using approximately 150 lines of code, enabling tractable expression of complex workloads with efficient mapping to spatial hardware. MLIR-AIR transforms high-level structured control flow into spatial programs that efficiently utilize the compute fabric and memory hierarchy of an NPU, leveraging asynchronous execution, tiling, and communication overlap through compiler-managed scheduling. |
| title | From Loop Nests to Silicon: Mapping AI Workloads onto AMD NPUs with MLIR-AIR |
| topic | Computation and Language Hardware Architecture Machine Learning |
| url | https://arxiv.org/abs/2510.14871 |