CD-NGP: A Fast Scalable Continual Representation for Dynamic Scenes
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911065141411840 |
|---|---|
| author | Liu, Zhenhuan Liu, Shuai Ning, Zhiwei Yang, Jie Zuo, Yifan Fang, Yuming Liu, Wei |
| author_facet | Liu, Zhenhuan Liu, Shuai Ning, Zhiwei Yang, Jie Zuo, Yifan Fang, Yuming Liu, Wei |
| contents | Novel view synthesis (NVS) in dynamic scenes faces persistent challenges in memory consumption, model complexity, training efficiency, and rendering quality. Offline methods offer high fidelity but suffer from high memory usage and limited scalability, while online approaches often trade quality for speed and compactness. We propose Continual Dynamic Neural Graphics Primitives (CD-NGP), a continual learning framework that reduces memory overhead and enhances scalability through parameter reuse. To avoid feature interference in dynamic scenes and improve rendering quality, our method combines spatial and temporal hash encodings, which compactly represent scene structures and motion patterns. We also introduce a new dataset comprising multi-view, long-duration ($>1200$ frames) videos with both rigid and non-rigid motion, which is not found in existing benchmarks. CD-NGP is evaluated on public datasets and our long video dataset, demonstrating superior scalability and reconstruction quality. It significantly reduces training memory usage to <14GB and requires only 0.4MB/frame in streaming bandwidth on DyNeRF -- substantially lower than most online baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_05166 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CD-NGP: A Fast Scalable Continual Representation for Dynamic Scenes Liu, Zhenhuan Liu, Shuai Ning, Zhiwei Yang, Jie Zuo, Yifan Fang, Yuming Liu, Wei Computer Vision and Pattern Recognition Novel view synthesis (NVS) in dynamic scenes faces persistent challenges in memory consumption, model complexity, training efficiency, and rendering quality. Offline methods offer high fidelity but suffer from high memory usage and limited scalability, while online approaches often trade quality for speed and compactness. We propose Continual Dynamic Neural Graphics Primitives (CD-NGP), a continual learning framework that reduces memory overhead and enhances scalability through parameter reuse. To avoid feature interference in dynamic scenes and improve rendering quality, our method combines spatial and temporal hash encodings, which compactly represent scene structures and motion patterns. We also introduce a new dataset comprising multi-view, long-duration ($>1200$ frames) videos with both rigid and non-rigid motion, which is not found in existing benchmarks. CD-NGP is evaluated on public datasets and our long video dataset, demonstrating superior scalability and reconstruction quality. It significantly reduces training memory usage to <14GB and requires only 0.4MB/frame in streaming bandwidth on DyNeRF -- substantially lower than most online baselines. |
| title | CD-NGP: A Fast Scalable Continual Representation for Dynamic Scenes |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.05166 |