SWiT-4D: Sliding-Window Transformer for Lossless and Parameter-Free Temporal 4D Generation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915669714403328 |
|---|---|
| author | Gong, Kehong Wen, Zhengyu Xu, Mingxi He, Weixia Wang, Qi Zhang, Ning Li, Zhengyu Li, Chenbin Lian, Dongze Zhao, Wei He, Xiaoyu Zhang, Mingyuan |
| author_facet | Gong, Kehong Wen, Zhengyu Xu, Mingxi He, Weixia Wang, Qi Zhang, Ning Li, Zhengyu Li, Chenbin Lian, Dongze Zhao, Wei He, Xiaoyu Zhang, Mingyuan |
| contents | Despite significant progress in 4D content generation, the conversion of monocular videos into high-quality animated 3D assets with explicit 4D meshes remains considerably challenging. The scarcity of large-scale, naturally captured 4D mesh datasets further limits the ability to train generalizable video-to-4D models from scratch in a purely data-driven manner. Meanwhile, advances in image-to-3D generation, supported by extensive datasets, offer powerful prior models that can be leveraged. To better utilize these priors while minimizing reliance on 4D supervision, we introduce SWiT-4D, a Sliding-Window Transformer for lossless, parameter-free temporal 4D mesh generation. SWiT-4D integrates seamlessly with any Diffusion Transformer (DiT)-based image-to-3D generator, adding spatial-temporal modeling across video frames while preserving the original single-image forward process, enabling 4D mesh reconstruction from videos of arbitrary length. To recover global translation, we further introduce an optimization-based trajectory module tailored for static-camera monocular videos. SWiT-4D demonstrates strong data efficiency: with only a single short (<10s) video for fine-tuning, it achieves high-fidelity geometry and stable temporal consistency, indicating practical deployability under extremely limited 4D supervision. Comprehensive experiments on both in-domain zoo-test sets and challenging out-of-domain benchmarks (C4D, Objaverse, and in-the-wild videos) show that SWiT-4D consistently outperforms existing baselines in temporal smoothness. Project page: https://animotionlab.github.io/SWIT4D/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_10860 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SWiT-4D: Sliding-Window Transformer for Lossless and Parameter-Free Temporal 4D Generation Gong, Kehong Wen, Zhengyu Xu, Mingxi He, Weixia Wang, Qi Zhang, Ning Li, Zhengyu Li, Chenbin Lian, Dongze Zhao, Wei He, Xiaoyu Zhang, Mingyuan Computer Vision and Pattern Recognition Despite significant progress in 4D content generation, the conversion of monocular videos into high-quality animated 3D assets with explicit 4D meshes remains considerably challenging. The scarcity of large-scale, naturally captured 4D mesh datasets further limits the ability to train generalizable video-to-4D models from scratch in a purely data-driven manner. Meanwhile, advances in image-to-3D generation, supported by extensive datasets, offer powerful prior models that can be leveraged. To better utilize these priors while minimizing reliance on 4D supervision, we introduce SWiT-4D, a Sliding-Window Transformer for lossless, parameter-free temporal 4D mesh generation. SWiT-4D integrates seamlessly with any Diffusion Transformer (DiT)-based image-to-3D generator, adding spatial-temporal modeling across video frames while preserving the original single-image forward process, enabling 4D mesh reconstruction from videos of arbitrary length. To recover global translation, we further introduce an optimization-based trajectory module tailored for static-camera monocular videos. SWiT-4D demonstrates strong data efficiency: with only a single short (<10s) video for fine-tuning, it achieves high-fidelity geometry and stable temporal consistency, indicating practical deployability under extremely limited 4D supervision. Comprehensive experiments on both in-domain zoo-test sets and challenging out-of-domain benchmarks (C4D, Objaverse, and in-the-wild videos) show that SWiT-4D consistently outperforms existing baselines in temporal smoothness. Project page: https://animotionlab.github.io/SWIT4D/ |
| title | SWiT-4D: Sliding-Window Transformer for Lossless and Parameter-Free Temporal 4D Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.10860 |