FB-4D: Spatial-Temporal Coherent Dynamic 3D Content Generation with Feature Banks
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908285178740736 |
|---|---|
| author | Li, Jinwei Gao, Huan-ang Li, Wenyi Chi, Haohan Liu, Chenyu Du, Chenxi Liu, Yiqian Gao, Mingju Zhang, Guiyu Zhang, Zongzheng Yi, Li Yao, Yao Zhao, Jingwei Li, Hongyang Wang, Yikai Zhao, Hao |
| author_facet | Li, Jinwei Gao, Huan-ang Li, Wenyi Chi, Haohan Liu, Chenyu Du, Chenxi Liu, Yiqian Gao, Mingju Zhang, Guiyu Zhang, Zongzheng Yi, Li Yao, Yao Zhao, Jingwei Li, Hongyang Wang, Yikai Zhao, Hao |
| contents | With the rapid advancements in diffusion models and 3D generation techniques, dynamic 3D content generation has become a crucial research area. However, achieving high-fidelity 4D (dynamic 3D) generation with strong spatial-temporal consistency remains a challenging task. Inspired by recent findings that pretrained diffusion features capture rich correspondences, we propose FB-4D, a novel 4D generation framework that integrates a Feature Bank mechanism to enhance both spatial and temporal consistency in generated frames. In FB-4D, we store features extracted from previous frames and fuse them into the process of generating subsequent frames, ensuring consistent characteristics across both time and multiple views. To ensure a compact representation, the Feature Bank is updated by a proposed dynamic merging mechanism. Leveraging this Feature Bank, we demonstrate for the first time that generating additional reference sequences through multiple autoregressive iterations can continuously improve generation performance. Experimental results show that FB-4D significantly outperforms existing methods in terms of rendering quality, spatial-temporal consistency, and robustness. It surpasses all multi-view generation tuning-free approaches by a large margin and achieves performance on par with training-based methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_20784 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FB-4D: Spatial-Temporal Coherent Dynamic 3D Content Generation with Feature Banks Li, Jinwei Gao, Huan-ang Li, Wenyi Chi, Haohan Liu, Chenyu Du, Chenxi Liu, Yiqian Gao, Mingju Zhang, Guiyu Zhang, Zongzheng Yi, Li Yao, Yao Zhao, Jingwei Li, Hongyang Wang, Yikai Zhao, Hao Computer Vision and Pattern Recognition With the rapid advancements in diffusion models and 3D generation techniques, dynamic 3D content generation has become a crucial research area. However, achieving high-fidelity 4D (dynamic 3D) generation with strong spatial-temporal consistency remains a challenging task. Inspired by recent findings that pretrained diffusion features capture rich correspondences, we propose FB-4D, a novel 4D generation framework that integrates a Feature Bank mechanism to enhance both spatial and temporal consistency in generated frames. In FB-4D, we store features extracted from previous frames and fuse them into the process of generating subsequent frames, ensuring consistent characteristics across both time and multiple views. To ensure a compact representation, the Feature Bank is updated by a proposed dynamic merging mechanism. Leveraging this Feature Bank, we demonstrate for the first time that generating additional reference sequences through multiple autoregressive iterations can continuously improve generation performance. Experimental results show that FB-4D significantly outperforms existing methods in terms of rendering quality, spatial-temporal consistency, and robustness. It surpasses all multi-view generation tuning-free approaches by a large margin and achieves performance on par with training-based methods. |
| title | FB-4D: Spatial-Temporal Coherent Dynamic 3D Content Generation with Feature Banks |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.20784 |