VTC: DNN Compilation with Virtual Tensors for Data Movement Elimination
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913022042177536 |
|---|---|
| author | Hu, Muyan Gupta, Ahan Yuan, Jiachen Gupta, Vima Kim, Taeksang Xu, Xin Kulkarni, Janardhan Dekel, Ofer Adve, Vikram Mendis, Charith |
| author_facet | Hu, Muyan Gupta, Ahan Yuan, Jiachen Gupta, Vima Kim, Taeksang Xu, Xin Kulkarni, Janardhan Dekel, Ofer Adve, Vikram Mendis, Charith |
| contents | With the widening gap between compute and memory operation latencies, data movement optimizations have become increasingly important for DNN compilation. Current optimizations such as layout transformations and operator fusion only target a subset of tensor operators and consequently miss important opportunities for reducing data movement in contemporary DNN workloads, including large language models.
We introduce VTC, a novel tensor compilation framework that for the first time eliminates all unnecessary data movement by targeting the full spectrum of data movement operators. VTC proposes the concept of virtual tensors to track data movement between compute operators via index mappings rather than expensive physical data transfers to and from global memory, which can seamlessly interoperate with existing computation kernels and handle arbitrary tensor operator compositions. We also introduce a novel data movement elimination algorithm to automatically identify a profitable virtual tensor creation strategy. Evaluation on a variety of DNNs shows that VTC can outperform existing ML compilers by up to 1.93x (1.28x on average) on NVIDIA GPUs with up to 60% (17.5% on average) inference memory savings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_09558 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | VTC: DNN Compilation with Virtual Tensors for Data Movement Elimination Hu, Muyan Gupta, Ahan Yuan, Jiachen Gupta, Vima Kim, Taeksang Xu, Xin Kulkarni, Janardhan Dekel, Ofer Adve, Vikram Mendis, Charith Distributed, Parallel, and Cluster Computing Machine Learning Programming Languages With the widening gap between compute and memory operation latencies, data movement optimizations have become increasingly important for DNN compilation. Current optimizations such as layout transformations and operator fusion only target a subset of tensor operators and consequently miss important opportunities for reducing data movement in contemporary DNN workloads, including large language models. We introduce VTC, a novel tensor compilation framework that for the first time eliminates all unnecessary data movement by targeting the full spectrum of data movement operators. VTC proposes the concept of virtual tensors to track data movement between compute operators via index mappings rather than expensive physical data transfers to and from global memory, which can seamlessly interoperate with existing computation kernels and handle arbitrary tensor operator compositions. We also introduce a novel data movement elimination algorithm to automatically identify a profitable virtual tensor creation strategy. Evaluation on a variety of DNNs shows that VTC can outperform existing ML compilers by up to 1.93x (1.28x on average) on NVIDIA GPUs with up to 60% (17.5% on average) inference memory savings. |
| title | VTC: DNN Compilation with Virtual Tensors for Data Movement Elimination |
| topic | Distributed, Parallel, and Cluster Computing Machine Learning Programming Languages |
| url | https://arxiv.org/abs/2604.09558 |