Consistent Instance Field for Dynamic Scene Understanding
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911322080280576 |
|---|---|
| author | Wu, Junyi Nguyen, Van Nguyen Planche, Benjamin Tao, Jiachen Sun, Changchang Gao, Zhongpai Zhao, Zhenghao Choudhuri, Anwesa Zhang, Gengyu Zheng, Meng Wang, Feiran Chen, Terrence Yan, Yan Wu, Ziyan |
| author_facet | Wu, Junyi Nguyen, Van Nguyen Planche, Benjamin Tao, Jiachen Sun, Changchang Gao, Zhongpai Zhao, Zhenghao Choudhuri, Anwesa Zhang, Gengyu Zheng, Meng Wang, Feiran Chen, Terrence Yan, Yan Wu, Ziyan |
| contents | We introduce Consistent Instance Field, a continuous and probabilistic spatio-temporal representation for dynamic scene understanding. Unlike prior methods that rely on discrete tracking or view-dependent features, our approach disentangles visibility from persistent object identity by modeling each space-time point with an occupancy probability and a conditional instance distribution. To realize this, we introduce a novel instance-embedded representation based on deformable 3D Gaussians, which jointly encode radiance and semantic information and are learned directly from input RGB images and instance masks through differentiable rasterization. Furthermore, we introduce new mechanisms to calibrate per-Gaussian identities and resample Gaussians toward semantically active regions, ensuring consistent instance representations across space and time. Experiments on HyperNeRF and Neu3D datasets demonstrate that our method significantly outperforms state-of-the-art methods on novel-view panoptic segmentation and open-vocabulary 4D querying tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_14126 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Consistent Instance Field for Dynamic Scene Understanding Wu, Junyi Nguyen, Van Nguyen Planche, Benjamin Tao, Jiachen Sun, Changchang Gao, Zhongpai Zhao, Zhenghao Choudhuri, Anwesa Zhang, Gengyu Zheng, Meng Wang, Feiran Chen, Terrence Yan, Yan Wu, Ziyan Computer Vision and Pattern Recognition We introduce Consistent Instance Field, a continuous and probabilistic spatio-temporal representation for dynamic scene understanding. Unlike prior methods that rely on discrete tracking or view-dependent features, our approach disentangles visibility from persistent object identity by modeling each space-time point with an occupancy probability and a conditional instance distribution. To realize this, we introduce a novel instance-embedded representation based on deformable 3D Gaussians, which jointly encode radiance and semantic information and are learned directly from input RGB images and instance masks through differentiable rasterization. Furthermore, we introduce new mechanisms to calibrate per-Gaussian identities and resample Gaussians toward semantically active regions, ensuring consistent instance representations across space and time. Experiments on HyperNeRF and Neu3D datasets demonstrate that our method significantly outperforms state-of-the-art methods on novel-view panoptic segmentation and open-vocabulary 4D querying tasks. |
| title | Consistent Instance Field for Dynamic Scene Understanding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.14126 |