NormalCrafter: Learning Temporally Consistent Normals from Video Diffusion Priors
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910911995838464 |
|---|---|
| author | Bin, Yanrui Hu, Wenbo Wang, Haoyuan Chen, Xinya Wang, Bing |
| author_facet | Bin, Yanrui Hu, Wenbo Wang, Haoyuan Chen, Xinya Wang, Bing |
| contents | Surface normal estimation serves as a cornerstone for a spectrum of computer vision applications. While numerous efforts have been devoted to static image scenarios, ensuring temporal coherence in video-based normal estimation remains a formidable challenge. Instead of merely augmenting existing methods with temporal components, we present NormalCrafter to leverage the inherent temporal priors of video diffusion models. To secure high-fidelity normal estimation across sequences, we propose Semantic Feature Regularization (SFR), which aligns diffusion features with semantic cues, encouraging the model to concentrate on the intrinsic semantics of the scene. Moreover, we introduce a two-stage training protocol that leverages both latent and pixel space learning to preserve spatial accuracy while maintaining long temporal context. Extensive evaluations demonstrate the efficacy of our method, showcasing a superior performance in generating temporally consistent normal sequences with intricate details from diverse videos. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_11427 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | NormalCrafter: Learning Temporally Consistent Normals from Video Diffusion Priors Bin, Yanrui Hu, Wenbo Wang, Haoyuan Chen, Xinya Wang, Bing Computer Vision and Pattern Recognition Surface normal estimation serves as a cornerstone for a spectrum of computer vision applications. While numerous efforts have been devoted to static image scenarios, ensuring temporal coherence in video-based normal estimation remains a formidable challenge. Instead of merely augmenting existing methods with temporal components, we present NormalCrafter to leverage the inherent temporal priors of video diffusion models. To secure high-fidelity normal estimation across sequences, we propose Semantic Feature Regularization (SFR), which aligns diffusion features with semantic cues, encouraging the model to concentrate on the intrinsic semantics of the scene. Moreover, we introduce a two-stage training protocol that leverages both latent and pixel space learning to preserve spatial accuracy while maintaining long temporal context. Extensive evaluations demonstrate the efficacy of our method, showcasing a superior performance in generating temporally consistent normal sequences with intricate details from diverse videos. |
| title | NormalCrafter: Learning Temporally Consistent Normals from Video Diffusion Priors |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.11427 |