Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.21986 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917191221248000 |
|---|---|
| author | Park, Dogyun Haji-Ali, Moayed Li, Yanyu Menapace, Willi Tulyakov, Sergey Kim, Hyunwoo J. Siarohin, Aliaksandr Kag, Anil |
| author_facet | Park, Dogyun Haji-Ali, Moayed Li, Yanyu Menapace, Willi Tulyakov, Sergey Kim, Hyunwoo J. Siarohin, Aliaksandr Kag, Anil |
| contents | Diffusion Transformers (DiTs) deliver state-of-the-art generative performance but their quadratic training cost with sequence length makes large-scale pretraining prohibitively expensive. Token dropping can reduce training cost, yet naïve strategies degrade representations, and existing methods are either parameter-heavy or fail at high drop ratios. We present SPRINT, Sparse--Dense Residual Fusion for Efficient Diffusion Transformers, a simple method that enables aggressive token dropping (up to 75%) while preserving quality. SPRINT leverages the complementary roles of shallow and deep layers: early layers process all tokens to capture local detail, deeper layers operate on a sparse subset to cut computation, and their outputs are fused through residual connections. Training follows a two-stage schedule: long masked pre-training for efficiency followed by short full-token fine-tuning to close the train--inference gap. On ImageNet-1K 256x256, SPRINT achieves 9.8x training savings with comparable FID/FDD, and at inference, its Path-Drop Guidance (PDG) nearly halves FLOPs while improving quality. These results establish SPRINT as a simple, effective, and general solution for efficient DiT training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_21986 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Sprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformers Park, Dogyun Haji-Ali, Moayed Li, Yanyu Menapace, Willi Tulyakov, Sergey Kim, Hyunwoo J. Siarohin, Aliaksandr Kag, Anil Computer Vision and Pattern Recognition Diffusion Transformers (DiTs) deliver state-of-the-art generative performance but their quadratic training cost with sequence length makes large-scale pretraining prohibitively expensive. Token dropping can reduce training cost, yet naïve strategies degrade representations, and existing methods are either parameter-heavy or fail at high drop ratios. We present SPRINT, Sparse--Dense Residual Fusion for Efficient Diffusion Transformers, a simple method that enables aggressive token dropping (up to 75%) while preserving quality. SPRINT leverages the complementary roles of shallow and deep layers: early layers process all tokens to capture local detail, deeper layers operate on a sparse subset to cut computation, and their outputs are fused through residual connections. Training follows a two-stage schedule: long masked pre-training for efficiency followed by short full-token fine-tuning to close the train--inference gap. On ImageNet-1K 256x256, SPRINT achieves 9.8x training savings with comparable FID/FDD, and at inference, its Path-Drop Guidance (PDG) nearly halves FLOPs while improving quality. These results establish SPRINT as a simple, effective, and general solution for efficient DiT training. |
| title | Sprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformers |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.21986 |