H3AE: High Compression, High Speed, and High Quality AutoEncoder for Video Diffusion Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915526724288512 |
|---|---|
| author | Wu, Yushu Li, Yanyu Skorokhodov, Ivan Kag, Anil Menapace, Willi Girish, Sharath Siarohin, Aliaksandr Wang, Yanzhi Tulyakov, Sergey |
| author_facet | Wu, Yushu Li, Yanyu Skorokhodov, Ivan Kag, Anil Menapace, Willi Girish, Sharath Siarohin, Aliaksandr Wang, Yanzhi Tulyakov, Sergey |
| contents | Autoencoder (AE) is the key to the success of latent diffusion models for image and video generation, reducing the denoising resolution and improving efficiency. However, the power of AE has long been underexplored in terms of network design, compression ratio, and training strategy. In this work, we systematically examine the architecture design choices and optimize the computation distribution to obtain a series of efficient and high-compression video AEs that can decode in real time even on mobile devices. We also propose an omni-training objective to unify the design of plain Autoencoder and image-conditioned I2V VAE, achieving multifunctionality in a single VAE network but with enhanced quality. In addition, we propose a novel latent consistency loss that provides stable improvements in reconstruction quality. Latent consistency loss outperforms prior auxiliary losses including LPIPS, GAN and DWT in terms of both quality improvements and simplicity. H3AE achieves ultra-high compression ratios and real-time decoding speed on GPU and mobile, and outperforms prior arts in terms of reconstruction metrics by a large margin. We finally validate our AE by training a DiT on its latent space and demonstrate fast, high-quality text-to-video generation capability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_10567 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | H3AE: High Compression, High Speed, and High Quality AutoEncoder for Video Diffusion Models Wu, Yushu Li, Yanyu Skorokhodov, Ivan Kag, Anil Menapace, Willi Girish, Sharath Siarohin, Aliaksandr Wang, Yanzhi Tulyakov, Sergey Computer Vision and Pattern Recognition Image and Video Processing Autoencoder (AE) is the key to the success of latent diffusion models for image and video generation, reducing the denoising resolution and improving efficiency. However, the power of AE has long been underexplored in terms of network design, compression ratio, and training strategy. In this work, we systematically examine the architecture design choices and optimize the computation distribution to obtain a series of efficient and high-compression video AEs that can decode in real time even on mobile devices. We also propose an omni-training objective to unify the design of plain Autoencoder and image-conditioned I2V VAE, achieving multifunctionality in a single VAE network but with enhanced quality. In addition, we propose a novel latent consistency loss that provides stable improvements in reconstruction quality. Latent consistency loss outperforms prior auxiliary losses including LPIPS, GAN and DWT in terms of both quality improvements and simplicity. H3AE achieves ultra-high compression ratios and real-time decoding speed on GPU and mobile, and outperforms prior arts in terms of reconstruction metrics by a large margin. We finally validate our AE by training a DiT on its latent space and demonstrate fast, high-quality text-to-video generation capability. |
| title | H3AE: High Compression, High Speed, and High Quality AutoEncoder for Video Diffusion Models |
| topic | Computer Vision and Pattern Recognition Image and Video Processing |
| url | https://arxiv.org/abs/2504.10567 |