QiMeng-TensorOp: Automatically Generating High-Performance Tensor Operators with Hardware Primitives
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910935587749888 |
|---|---|
| author | Zhang, Xuzhi Peng, Shaohui Zhou, Qirui Wen, Yuanbo Guo, Qi Chen, Ruizhi Zhu, Xinguo Xiong, Weiqiang Chen, Haixin Ma, Congying Gao, Ke Zhao, Chen Wu, Yanjun Chen, Yunji Li, Ling |
| author_facet | Zhang, Xuzhi Peng, Shaohui Zhou, Qirui Wen, Yuanbo Guo, Qi Chen, Ruizhi Zhu, Xinguo Xiong, Weiqiang Chen, Haixin Ma, Congying Gao, Ke Zhao, Chen Wu, Yanjun Chen, Yunji Li, Ling |
| contents | Computation-intensive tensor operators constitute over 90\% of the computations in Large Language Models (LLMs) and Deep Neural Networks.Automatically and efficiently generating high-performance tensor operators with hardware primitives is crucial for diverse and ever-evolving hardware architectures like RISC-V, ARM, and GPUs, as manually optimized implementation takes at least months and lacks portability.LLMs excel at generating high-level language codes, but they struggle to fully comprehend hardware characteristics and produce high-performance tensor operators. We introduce a tensor-operator auto-generation framework with a one-line user prompt (QiMeng-TensorOp), which enables LLMs to automatically exploit hardware characteristics to generate tensor operators with hardware primitives, and tune parameters for optimal performance across diverse hardware. Experimental results on various hardware platforms, SOTA LLMs, and typical tensor operators demonstrate that QiMeng-TensorOp effectively unleashes the computing capability of various hardware platforms, and automatically generates tensor operators of superior performance. Compared with vanilla LLMs, QiMeng-TensorOp achieves up to $1291 \times$ performance improvement. Even compared with human experts, QiMeng-TensorOp could reach $251 \%$ of OpenBLAS on RISC-V CPUs, and $124 \%$ of cuBLAS on NVIDIA GPUs. Additionally, QiMeng-TensorOp also significantly reduces development costs by $200 \times$ compared with human experts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_06302 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | QiMeng-TensorOp: Automatically Generating High-Performance Tensor Operators with Hardware Primitives Zhang, Xuzhi Peng, Shaohui Zhou, Qirui Wen, Yuanbo Guo, Qi Chen, Ruizhi Zhu, Xinguo Xiong, Weiqiang Chen, Haixin Ma, Congying Gao, Ke Zhao, Chen Wu, Yanjun Chen, Yunji Li, Ling Machine Learning Artificial Intelligence I.2.2 Computation-intensive tensor operators constitute over 90\% of the computations in Large Language Models (LLMs) and Deep Neural Networks.Automatically and efficiently generating high-performance tensor operators with hardware primitives is crucial for diverse and ever-evolving hardware architectures like RISC-V, ARM, and GPUs, as manually optimized implementation takes at least months and lacks portability.LLMs excel at generating high-level language codes, but they struggle to fully comprehend hardware characteristics and produce high-performance tensor operators. We introduce a tensor-operator auto-generation framework with a one-line user prompt (QiMeng-TensorOp), which enables LLMs to automatically exploit hardware characteristics to generate tensor operators with hardware primitives, and tune parameters for optimal performance across diverse hardware. Experimental results on various hardware platforms, SOTA LLMs, and typical tensor operators demonstrate that QiMeng-TensorOp effectively unleashes the computing capability of various hardware platforms, and automatically generates tensor operators of superior performance. Compared with vanilla LLMs, QiMeng-TensorOp achieves up to $1291 \times$ performance improvement. Even compared with human experts, QiMeng-TensorOp could reach $251 \%$ of OpenBLAS on RISC-V CPUs, and $124 \%$ of cuBLAS on NVIDIA GPUs. Additionally, QiMeng-TensorOp also significantly reduces development costs by $200 \times$ compared with human experts. |
| title | QiMeng-TensorOp: Automatically Generating High-Performance Tensor Operators with Hardware Primitives |
| topic | Machine Learning Artificial Intelligence I.2.2 |
| url | https://arxiv.org/abs/2505.06302 |