Streaming Tensor Programs: A Streaming Abstraction for Dynamic Parallelism
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915758373601280 |
|---|---|
| author | Sohn, Gina Zhang, Genghan Hossfeld, Konstantin Kim, Jungwoo Sobotka, Nathan Zhang, Nathan Hsu, Olivia Olukotun, Kunle |
| author_facet | Sohn, Gina Zhang, Genghan Hossfeld, Konstantin Kim, Jungwoo Sobotka, Nathan Zhang, Nathan Hsu, Olivia Olukotun, Kunle |
| contents | Dynamic behaviors are becoming prevalent in tensor applications, like machine learning, where many widely used models contain data-dependent tensor shapes and control flow. However, the limited expressiveness of prior programming abstractions for spatial dataflow accelerators (SDAs) forces these dynamic behaviors to be implemented statically and/or unoptimized. To address these challenges, we present Streaming Tensor Programs (STeP), a streaming abstraction that enables dynamic tensor workloads to run efficiently on SDAs. STeP introduces flexible routing operators, an explicit memory hierarchy, and symbolic-shape semantics that expose dynamic data rates and tensor dimensions. These capabilities unlock new optimizations, like dynamic tiling, dynamic parallelization, and configuration time-multiplexing, that adapt SDA execution to dynamic behaviors while preserving dataflow efficiency. Using a cycle-approximate simulator on representative LLM layers and a full model with real-world traces, STeP enables: dynamic tiling that breaks the Pareto-optimal frontier from prior work, dynamic parallelization that improves latency by ~2.72x, and configuration time-multiplexing that increases compute utilization by ~2.64x over prior SDA abstractions and their implementations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_07776 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Streaming Tensor Programs: A Streaming Abstraction for Dynamic Parallelism Sohn, Gina Zhang, Genghan Hossfeld, Konstantin Kim, Jungwoo Sobotka, Nathan Zhang, Nathan Hsu, Olivia Olukotun, Kunle Programming Languages Hardware Architecture Machine Learning Dynamic behaviors are becoming prevalent in tensor applications, like machine learning, where many widely used models contain data-dependent tensor shapes and control flow. However, the limited expressiveness of prior programming abstractions for spatial dataflow accelerators (SDAs) forces these dynamic behaviors to be implemented statically and/or unoptimized. To address these challenges, we present Streaming Tensor Programs (STeP), a streaming abstraction that enables dynamic tensor workloads to run efficiently on SDAs. STeP introduces flexible routing operators, an explicit memory hierarchy, and symbolic-shape semantics that expose dynamic data rates and tensor dimensions. These capabilities unlock new optimizations, like dynamic tiling, dynamic parallelization, and configuration time-multiplexing, that adapt SDA execution to dynamic behaviors while preserving dataflow efficiency. Using a cycle-approximate simulator on representative LLM layers and a full model with real-world traces, STeP enables: dynamic tiling that breaks the Pareto-optimal frontier from prior work, dynamic parallelization that improves latency by ~2.72x, and configuration time-multiplexing that increases compute utilization by ~2.64x over prior SDA abstractions and their implementations. |
| title | Streaming Tensor Programs: A Streaming Abstraction for Dynamic Parallelism |
| topic | Programming Languages Hardware Architecture Machine Learning |
| url | https://arxiv.org/abs/2511.07776 |