Library Liberation: Competitive Performance Matmul Through Compiler-composed Nanokernels
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912715403952128 |
|---|---|
| author | Thangamani, Arun Shahid, Md Asghar Ahmad Siemieniuk, Adam Morel, Rolf Golin, Renato Heinecke, Alexander |
| author_facet | Thangamani, Arun Shahid, Md Asghar Ahmad Siemieniuk, Adam Morel, Rolf Golin, Renato Heinecke, Alexander |
| contents | The rapidly evolving landscape of AI and machine learning workloads has widened the gap between high-level domain operations and efficient hardware utilization. Achieving near-peak performance still demands deep hardware expertise-experts either handcraft target-specific kernels (e.g., DeepSeek) or rely on specialized libraries (e.g., CUTLASS)-both of which add complexity and limit scalability for most ML practitioners.
This paper introduces a compilation scheme that automatically generates scalable, high-performance microkernels by leveraging the MLIR dialects to bridge domain-level operations and processor capabilities. Our approach removes dependence on low-level libraries by enabling the compiler to auto-generate near-optimal code directly. At its core is a mechanism for composing nanokernels from low-level IR constructs with near-optimal register utilization, forming efficient microkernels tailored to each target. We implement this technique in an MLIR-based compiler supporting both vector and tile based CPU instructions. Experiments show that the generated nanokernels are of production-quality, and competitive with state-of-the-art microkernel libraries. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_13764 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Library Liberation: Competitive Performance Matmul Through Compiler-composed Nanokernels Thangamani, Arun Shahid, Md Asghar Ahmad Siemieniuk, Adam Morel, Rolf Golin, Renato Heinecke, Alexander Machine Learning Performance Programming Languages Software Engineering The rapidly evolving landscape of AI and machine learning workloads has widened the gap between high-level domain operations and efficient hardware utilization. Achieving near-peak performance still demands deep hardware expertise-experts either handcraft target-specific kernels (e.g., DeepSeek) or rely on specialized libraries (e.g., CUTLASS)-both of which add complexity and limit scalability for most ML practitioners. This paper introduces a compilation scheme that automatically generates scalable, high-performance microkernels by leveraging the MLIR dialects to bridge domain-level operations and processor capabilities. Our approach removes dependence on low-level libraries by enabling the compiler to auto-generate near-optimal code directly. At its core is a mechanism for composing nanokernels from low-level IR constructs with near-optimal register utilization, forming efficient microkernels tailored to each target. We implement this technique in an MLIR-based compiler supporting both vector and tile based CPU instructions. Experiments show that the generated nanokernels are of production-quality, and competitive with state-of-the-art microkernel libraries. |
| title | Library Liberation: Competitive Performance Matmul Through Compiler-composed Nanokernels |
| topic | Machine Learning Performance Programming Languages Software Engineering |
| url | https://arxiv.org/abs/2511.13764 |