Taming Diffusion Transformer for Efficient Mobile Video Generation in Seconds
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908567800381440 |
|---|---|
| author | Wu, Yushu Li, Yanyu Kag, Anil Skorokhodov, Ivan Menapace, Willi Ma, Ke Sahni, Arpit Hu, Ju Siarohin, Aliaksandr Sagar, Dhritiman Wang, Yanzhi Tulyakov, Sergey |
| author_facet | Wu, Yushu Li, Yanyu Kag, Anil Skorokhodov, Ivan Menapace, Willi Ma, Ke Sahni, Arpit Hu, Ju Siarohin, Aliaksandr Sagar, Dhritiman Wang, Yanzhi Tulyakov, Sergey |
| contents | Diffusion Transformers (DiT) have shown strong performance in video generation tasks, but their high computational cost makes them impractical for resource-constrained devices like smartphones, and practical on-device generation is even more challenging. In this work, we propose a series of novel optimizations to significantly accelerate video generation and enable practical deployment on mobile platforms. First, we employ a highly compressed variational autoencoder (VAE) to reduce the dimensionality of the input data without sacrificing visual quality. Second, we introduce a KD-guided, sensitivity-aware tri-level pruning strategy to shrink the model size to suit mobile platforms while preserving critical performance characteristics. Third, we develop an adversarial step distillation technique tailored for DiT, which allows us to reduce the number of inference steps to four. Combined, these optimizations enable our model to achieve approximately 15 frames per second (FPS) generation speed on an iPhone 16 Pro Max, demonstrating the feasibility of efficient, high-quality video generation on mobile devices. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_13343 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Taming Diffusion Transformer for Efficient Mobile Video Generation in Seconds Wu, Yushu Li, Yanyu Kag, Anil Skorokhodov, Ivan Menapace, Willi Ma, Ke Sahni, Arpit Hu, Ju Siarohin, Aliaksandr Sagar, Dhritiman Wang, Yanzhi Tulyakov, Sergey Computer Vision and Pattern Recognition Image and Video Processing Diffusion Transformers (DiT) have shown strong performance in video generation tasks, but their high computational cost makes them impractical for resource-constrained devices like smartphones, and practical on-device generation is even more challenging. In this work, we propose a series of novel optimizations to significantly accelerate video generation and enable practical deployment on mobile platforms. First, we employ a highly compressed variational autoencoder (VAE) to reduce the dimensionality of the input data without sacrificing visual quality. Second, we introduce a KD-guided, sensitivity-aware tri-level pruning strategy to shrink the model size to suit mobile platforms while preserving critical performance characteristics. Third, we develop an adversarial step distillation technique tailored for DiT, which allows us to reduce the number of inference steps to four. Combined, these optimizations enable our model to achieve approximately 15 frames per second (FPS) generation speed on an iPhone 16 Pro Max, demonstrating the feasibility of efficient, high-quality video generation on mobile devices. |
| title | Taming Diffusion Transformer for Efficient Mobile Video Generation in Seconds |
| topic | Computer Vision and Pattern Recognition Image and Video Processing |
| url | https://arxiv.org/abs/2507.13343 |