FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909514963353600 |
|---|---|
| author | Anagnostidis, Sotiris Bachmann, Gregor Kim, Yeongmin Kohler, Jonas Georgopoulos, Markos Sanakoyeu, Artsiom Du, Yuming Pumarola, Albert Thabet, Ali Schönfeld, Edgar |
| author_facet | Anagnostidis, Sotiris Bachmann, Gregor Kim, Yeongmin Kohler, Jonas Georgopoulos, Markos Sanakoyeu, Artsiom Du, Yuming Pumarola, Albert Thabet, Ali Schönfeld, Edgar |
| contents | Despite their remarkable performance, modern Diffusion Transformers are hindered by substantial resource requirements during inference, stemming from the fixed and large amount of compute needed for each denoising step. In this work, we revisit the conventional static paradigm that allocates a fixed compute budget per denoising iteration and propose a dynamic strategy instead. Our simple and sample-efficient framework enables pre-trained DiT models to be converted into \emph{flexible} ones -- dubbed FlexiDiT -- allowing them to process inputs at varying compute budgets. We demonstrate how a single \emph{flexible} model can generate images without any drop in quality, while reducing the required FLOPs by more than $40$\% compared to their static counterparts, for both class-conditioned and text-conditioned image generation. Our method is general and agnostic to input and conditioning modalities. We show how our approach can be readily extended for video generation, where FlexiDiT models generate samples with up to $75$\% less compute without compromising performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_20126 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute Anagnostidis, Sotiris Bachmann, Gregor Kim, Yeongmin Kohler, Jonas Georgopoulos, Markos Sanakoyeu, Artsiom Du, Yuming Pumarola, Albert Thabet, Ali Schönfeld, Edgar Machine Learning Computer Vision and Pattern Recognition Despite their remarkable performance, modern Diffusion Transformers are hindered by substantial resource requirements during inference, stemming from the fixed and large amount of compute needed for each denoising step. In this work, we revisit the conventional static paradigm that allocates a fixed compute budget per denoising iteration and propose a dynamic strategy instead. Our simple and sample-efficient framework enables pre-trained DiT models to be converted into \emph{flexible} ones -- dubbed FlexiDiT -- allowing them to process inputs at varying compute budgets. We demonstrate how a single \emph{flexible} model can generate images without any drop in quality, while reducing the required FLOPs by more than $40$\% compared to their static counterparts, for both class-conditioned and text-conditioned image generation. Our method is general and agnostic to input and conditioning modalities. We show how our approach can be readily extended for video generation, where FlexiDiT models generate samples with up to $75$\% less compute without compromising performance. |
| title | FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute |
| topic | Machine Learning Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2502.20126 |