DeCo-VAE: Learning Compact Latents for Video Reconstruction via Decoupled Representation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917089364672512 |
|---|---|
| author | Yin, Xiangchen Yuan, Jiahui Hu, Zhangchi Sun, Wenzhang Chen, Jie Qiao, Xiaozhen Li, Hao Sun, Xiaoyan |
| author_facet | Yin, Xiangchen Yuan, Jiahui Hu, Zhangchi Sun, Wenzhang Chen, Jie Qiao, Xiaozhen Li, Hao Sun, Xiaoyan |
| contents | Existing video Variational Autoencoders (VAEs) generally overlook the similarity between frame contents, leading to redundant latent modeling. In this paper, we propose decoupled VAE (DeCo-VAE) to achieve compact latent representation. Instead of encoding RGB pixels directly, we decompose video content into distinct components via explicit decoupling: keyframe, motion and residual, and learn dedicated latent representation for each. To avoid cross-component interference, we design dedicated encoders for each decoupled component and adopt a shared 3D decoder to maintain spatiotemporal consistency during reconstruction. We further utilize a decoupled adaptation strategy that freezes partial encoders while training the others sequentially, ensuring stable training and accurate learning of both static and dynamic features. Extensive quantitative and qualitative experiments demonstrate that DeCo-VAE achieves superior video reconstruction performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_14530 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DeCo-VAE: Learning Compact Latents for Video Reconstruction via Decoupled Representation Yin, Xiangchen Yuan, Jiahui Hu, Zhangchi Sun, Wenzhang Chen, Jie Qiao, Xiaozhen Li, Hao Sun, Xiaoyan Computer Vision and Pattern Recognition Machine Learning Multimedia Existing video Variational Autoencoders (VAEs) generally overlook the similarity between frame contents, leading to redundant latent modeling. In this paper, we propose decoupled VAE (DeCo-VAE) to achieve compact latent representation. Instead of encoding RGB pixels directly, we decompose video content into distinct components via explicit decoupling: keyframe, motion and residual, and learn dedicated latent representation for each. To avoid cross-component interference, we design dedicated encoders for each decoupled component and adopt a shared 3D decoder to maintain spatiotemporal consistency during reconstruction. We further utilize a decoupled adaptation strategy that freezes partial encoders while training the others sequentially, ensuring stable training and accurate learning of both static and dynamic features. Extensive quantitative and qualitative experiments demonstrate that DeCo-VAE achieves superior video reconstruction performance. |
| title | DeCo-VAE: Learning Compact Latents for Video Reconstruction via Decoupled Representation |
| topic | Computer Vision and Pattern Recognition Machine Learning Multimedia |
| url | https://arxiv.org/abs/2511.14530 |