RedFuser: An Automatic Operator Fusion Framework for Cascaded Reductions on AI Accelerators
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911503954739200 |
|---|---|
| author | Tang, Xinsheng Li, Yangcheng Wang, Nan Shu, Zhiyi Ling, Xingyu Xing, Junna Zhou, Peng Liu, Qiang |
| author_facet | Tang, Xinsheng Li, Yangcheng Wang, Nan Shu, Zhiyi Ling, Xingyu Xing, Junna Zhou, Peng Liu, Qiang |
| contents | Operator fusion, as a key performance optimization technique in the deployment of AI models, significantly improves execution efficiency and has been widely adopted in modern AI compilers. However, for cascaded reduction operations involving multiple loops with inter-loop data dependencies, such as the safe softmax followed by GEMM within attention mechanisms, existing compilers lack effective automated fusion and kernel generation capabilities. Although some works have addressed specific instances through hand-crafted fusion strategies, their solutions are limited in generality and difficult to extend to other similar structures. Given the prevalence of such computational patterns in deep learning models, there remains significant untapped potential in achieving general and automated fusion optimization.
In this paper, we present a formal theoretical methodology for analyzing cascaded reductions which can fuse them into a single loop and introduce an incremental computation form. Based on this methodology, we design Reduction Fuser (RedFuser), a framework that automatically identifies supported cascaded reduction patterns and generates optimized fused kernels. Experiments show that RedFuser successfully fuses diverse workloads, achieving up to 2$\times$ to 5$\times$ speedup over state-of-the-art AI compilers and matching the performance of highly optimized hand-written kernels. The code is available at https://github.com/alibaba/redfuser |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_10026 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | RedFuser: An Automatic Operator Fusion Framework for Cascaded Reductions on AI Accelerators Tang, Xinsheng Li, Yangcheng Wang, Nan Shu, Zhiyi Ling, Xingyu Xing, Junna Zhou, Peng Liu, Qiang Hardware Architecture Artificial Intelligence Distributed, Parallel, and Cluster Computing Performance Operator fusion, as a key performance optimization technique in the deployment of AI models, significantly improves execution efficiency and has been widely adopted in modern AI compilers. However, for cascaded reduction operations involving multiple loops with inter-loop data dependencies, such as the safe softmax followed by GEMM within attention mechanisms, existing compilers lack effective automated fusion and kernel generation capabilities. Although some works have addressed specific instances through hand-crafted fusion strategies, their solutions are limited in generality and difficult to extend to other similar structures. Given the prevalence of such computational patterns in deep learning models, there remains significant untapped potential in achieving general and automated fusion optimization. In this paper, we present a formal theoretical methodology for analyzing cascaded reductions which can fuse them into a single loop and introduce an incremental computation form. Based on this methodology, we design Reduction Fuser (RedFuser), a framework that automatically identifies supported cascaded reduction patterns and generates optimized fused kernels. Experiments show that RedFuser successfully fuses diverse workloads, achieving up to 2$\times$ to 5$\times$ speedup over state-of-the-art AI compilers and matching the performance of highly optimized hand-written kernels. The code is available at https://github.com/alibaba/redfuser |
| title | RedFuser: An Automatic Operator Fusion Framework for Cascaded Reductions on AI Accelerators |
| topic | Hardware Architecture Artificial Intelligence Distributed, Parallel, and Cluster Computing Performance |
| url | https://arxiv.org/abs/2603.10026 |