4DStyleGaussian: Zero-shot 4D Style Transfer with Gaussian Splatting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910648961597440 |
|---|---|
| author | Liang, Wanlin Xu, Hongbin Chen, Weitao Xiao, Feng Kang, Wenxiong |
| author_facet | Liang, Wanlin Xu, Hongbin Chen, Weitao Xiao, Feng Kang, Wenxiong |
| contents | 3D neural style transfer has gained significant attention for its potential to provide user-friendly stylization with spatial consistency. However, existing 3D style transfer methods often fall short in terms of inference efficiency, generalization ability, and struggle to handle dynamic scenes with temporal consistency. In this paper, we introduce 4DStyleGaussian, a novel 4D style transfer framework designed to achieve real-time stylization of arbitrary style references while maintaining reasonable content affinity, multi-view consistency, and temporal coherence. Our approach leverages an embedded 4D Gaussian Splatting technique, which is trained using a reversible neural network for reducing content loss in the feature distillation process. Utilizing the 4D embedded Gaussians, we predict a 4D style transformation matrix that facilitates spatially and temporally consistent style transfer with Gaussian Splatting. Experiments demonstrate that our method can achieve high-quality and zero-shot stylization for 4D scenarios with enhanced efficiency and spatial-temporal consistency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_10412 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | 4DStyleGaussian: Zero-shot 4D Style Transfer with Gaussian Splatting Liang, Wanlin Xu, Hongbin Chen, Weitao Xiao, Feng Kang, Wenxiong Computer Vision and Pattern Recognition 3D neural style transfer has gained significant attention for its potential to provide user-friendly stylization with spatial consistency. However, existing 3D style transfer methods often fall short in terms of inference efficiency, generalization ability, and struggle to handle dynamic scenes with temporal consistency. In this paper, we introduce 4DStyleGaussian, a novel 4D style transfer framework designed to achieve real-time stylization of arbitrary style references while maintaining reasonable content affinity, multi-view consistency, and temporal coherence. Our approach leverages an embedded 4D Gaussian Splatting technique, which is trained using a reversible neural network for reducing content loss in the feature distillation process. Utilizing the 4D embedded Gaussians, we predict a 4D style transformation matrix that facilitates spatially and temporally consistent style transfer with Gaussian Splatting. Experiments demonstrate that our method can achieve high-quality and zero-shot stylization for 4D scenarios with enhanced efficiency and spatial-temporal consistency. |
| title | 4DStyleGaussian: Zero-shot 4D Style Transfer with Gaussian Splatting |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.10412 |