Sawtooth Wavefront Reordering: Enhanced CuTile FlashAttention on NVIDIA GB10
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912848349757440 |
|---|---|
| author | Zhu, Yifan Pan, Yekai Ding, Chen |
| author_facet | Zhu, Yifan Pan, Yekai Ding, Chen |
| contents | High-performance attention kernels are essential for Large Language Models. This paper presents analysis of CuTile-based Flash Attention memory behavior and a technique to improve its cache performance. In particular, our analysis on the NVIDIA GB10 (Grace Blackwell) identifies the main cause of L2 cache miss. Leveraging this insight, we introduce a new programming technique called Sawtooth Wavefront Reordering that reduces L2 misses. We validate it in both CUDA and CuTile, observing 50\% or greater reduction in L2 misses and up to 60\% increase in throughput on GB10. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_16032 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Sawtooth Wavefront Reordering: Enhanced CuTile FlashAttention on NVIDIA GB10 Zhu, Yifan Pan, Yekai Ding, Chen Performance Artificial Intelligence Machine Learning Operating Systems High-performance attention kernels are essential for Large Language Models. This paper presents analysis of CuTile-based Flash Attention memory behavior and a technique to improve its cache performance. In particular, our analysis on the NVIDIA GB10 (Grace Blackwell) identifies the main cause of L2 cache miss. Leveraging this insight, we introduce a new programming technique called Sawtooth Wavefront Reordering that reduces L2 misses. We validate it in both CUDA and CuTile, observing 50\% or greater reduction in L2 misses and up to 60\% increase in throughput on GB10. |
| title | Sawtooth Wavefront Reordering: Enhanced CuTile FlashAttention on NVIDIA GB10 |
| topic | Performance Artificial Intelligence Machine Learning Operating Systems |
| url | https://arxiv.org/abs/2601.16032 |