SATA: Sparsity-Aware Scheduling for Selective Token Attention
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914285996736512 |
|---|---|
| author | Fan, Zhenkun Wan, Zishen Liu, Che-Kai Lele, Ashwin Sanjay Khwa, Win-San Zhang, Bo Chang, Meng-Fan Raychowdhury, Arijit |
| author_facet | Fan, Zhenkun Wan, Zishen Liu, Che-Kai Lele, Ashwin Sanjay Khwa, Win-San Zhang, Bo Chang, Meng-Fan Raychowdhury, Arijit |
| contents | Transformers have become the foundation of numerous state-of-the-art AI models across diverse domains, thanks to their powerful attention mechanism for modeling long-range dependencies. However, the quadratic scaling complexity of attention poses significant challenges for efficient hardware implementation. While techniques such as quantization and pruning help mitigate this issue, selective token attention offers a promising alternative by narrowing the attention scope to only the most relevant tokens, reducing computation and filtering out noise.
In this work, we propose SATA, a locality-centric dynamic scheduling scheme that proactively manages sparsely distributed access patterns from selective Query-Key operations. By reordering operand flow and exploiting data locality, our approach enables early fetch and retirement of intermediate Query/Key vectors, improving system utilization. We implement and evaluate our token management strategy in a control and compute system, using runtime traces from selective-attention-based models. Experimental results show that our method improves system throughput by up to 1.76x and boosts energy efficiency by 2.94x, while incurring minimal scheduling overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_20267 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SATA: Sparsity-Aware Scheduling for Selective Token Attention Fan, Zhenkun Wan, Zishen Liu, Che-Kai Lele, Ashwin Sanjay Khwa, Win-San Zhang, Bo Chang, Meng-Fan Raychowdhury, Arijit Hardware Architecture Transformers have become the foundation of numerous state-of-the-art AI models across diverse domains, thanks to their powerful attention mechanism for modeling long-range dependencies. However, the quadratic scaling complexity of attention poses significant challenges for efficient hardware implementation. While techniques such as quantization and pruning help mitigate this issue, selective token attention offers a promising alternative by narrowing the attention scope to only the most relevant tokens, reducing computation and filtering out noise. In this work, we propose SATA, a locality-centric dynamic scheduling scheme that proactively manages sparsely distributed access patterns from selective Query-Key operations. By reordering operand flow and exploiting data locality, our approach enables early fetch and retirement of intermediate Query/Key vectors, improving system utilization. We implement and evaluate our token management strategy in a control and compute system, using runtime traces from selective-attention-based models. Experimental results show that our method improves system throughput by up to 1.76x and boosts energy efficiency by 2.94x, while incurring minimal scheduling overhead. |
| title | SATA: Sparsity-Aware Scheduling for Selective Token Attention |
| topic | Hardware Architecture |
| url | https://arxiv.org/abs/2601.20267 |