DITRON: Distributed Multi-level Tiling Compiler for Parallel Tensor Programs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914528457916416 |
|---|---|
| author | Zheng, Size Zheng, Xuegui Sun, Hanshi Hou, Qi Bao, Wenlei Li, Shiyu Duanmu, Haojie Fang, Jin Xue, Chenli Huang, Chenhui Liu, Yuanqiang Chen, Renze Zheng, Ningxin Wang, Dongyang Chang, Li-Wen Lu, Liqiang Liang, Yun Zhai, Jidong Liu, Xin |
| author_facet | Zheng, Size Zheng, Xuegui Sun, Hanshi Hou, Qi Bao, Wenlei Li, Shiyu Duanmu, Haojie Fang, Jin Xue, Chenli Huang, Chenhui Liu, Yuanqiang Chen, Renze Zheng, Ningxin Wang, Dongyang Chang, Li-Wen Lu, Liqiang Liang, Yun Zhai, Jidong Liu, Xin |
| contents | The scaling of large language models (LLMs) is currently bottlenecked by the rigidity of distributed programming. While high-performance libraries like CuBLAS and NCCL provide optimized primitives, they lack the flexibility required for rapidly evolving model architectures. Conversely, existing tensor compilers fail to address the complex memory hierarchy of distributed clusters effectively. To bridge this gap, we propose DITRON, a scalable tile-level compiler that democratizes high-performance distributed kernel development. DITRON introduces a novel hierarchical programming abstraction spanning Core, Device, and Task levels to map tensor programs efficiently onto heterogeneous distributed hardware. This abstraction allows DITRON to support diverse parallelism strategies while abstracting away the complexity of inter-node and intra-node communication.
Evaluated across large-scale clusters, DITRON achieves performance parity with or exceeding expert-tuned CUDA libraries, delivering speedups of $6\%-30\%$ on isolated kernels and $5\%-30\%$ on end-to-end inference in vLLM. Furthermore, DITRON demonstrates strong portability, achieving significant speedups on both NVIDIA and AMD platforms. \ours{} has been deployed at the enterprise level for both training and inference. It achieves an MFU improvement of over 10\% in training tasks, saving approximately 500,000 GPU hours of training cost per month. For inference tasks, it delivers an end-to-end gain of over 20\% and has been applied to cloud service inference and edge inference scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_02953 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DITRON: Distributed Multi-level Tiling Compiler for Parallel Tensor Programs Zheng, Size Zheng, Xuegui Sun, Hanshi Hou, Qi Bao, Wenlei Li, Shiyu Duanmu, Haojie Fang, Jin Xue, Chenli Huang, Chenhui Liu, Yuanqiang Chen, Renze Zheng, Ningxin Wang, Dongyang Chang, Li-Wen Lu, Liqiang Liang, Yun Zhai, Jidong Liu, Xin Programming Languages The scaling of large language models (LLMs) is currently bottlenecked by the rigidity of distributed programming. While high-performance libraries like CuBLAS and NCCL provide optimized primitives, they lack the flexibility required for rapidly evolving model architectures. Conversely, existing tensor compilers fail to address the complex memory hierarchy of distributed clusters effectively. To bridge this gap, we propose DITRON, a scalable tile-level compiler that democratizes high-performance distributed kernel development. DITRON introduces a novel hierarchical programming abstraction spanning Core, Device, and Task levels to map tensor programs efficiently onto heterogeneous distributed hardware. This abstraction allows DITRON to support diverse parallelism strategies while abstracting away the complexity of inter-node and intra-node communication. Evaluated across large-scale clusters, DITRON achieves performance parity with or exceeding expert-tuned CUDA libraries, delivering speedups of $6\%-30\%$ on isolated kernels and $5\%-30\%$ on end-to-end inference in vLLM. Furthermore, DITRON demonstrates strong portability, achieving significant speedups on both NVIDIA and AMD platforms. \ours{} has been deployed at the enterprise level for both training and inference. It achieves an MFU improvement of over 10\% in training tasks, saving approximately 500,000 GPU hours of training cost per month. For inference tasks, it delivers an end-to-end gain of over 20\% and has been applied to cloud service inference and edge inference scenarios. |
| title | DITRON: Distributed Multi-level Tiling Compiler for Parallel Tensor Programs |
| topic | Programming Languages |
| url | https://arxiv.org/abs/2605.02953 |