LiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Rendering and Control
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917952619544576 |
|---|---|
| author | Qu, Delin Chen, Qizhi Zhang, Pingrui Gao, Xianqiang Li, Junzhe Zhao, Bin Wang, Dong Li, Xuelong |
| author_facet | Qu, Delin Chen, Qizhi Zhang, Pingrui Gao, Xianqiang Li, Junzhe Zhao, Bin Wang, Dong Li, Xuelong |
| contents | This paper scales object-level reconstruction to complex scenes, advancing interactive scene reconstruction. We introduce two datasets, OmniSim and InterReal, featuring 28 scenes with multiple interactive objects. To tackle the challenge of inaccurate interactive motion recovery in complex scenes, we propose LiveScene, a scene-level language-embedded interactive radiance field that efficiently reconstructs and controls multiple objects. By decomposing the interactive scene into local deformable fields, LiveScene enables separate reconstruction of individual object motions, reducing memory consumption. Additionally, our interaction-aware language embedding localizes individual interactive objects, allowing for arbitrary control using natural language. Our approach demonstrates significant superiority in novel view synthesis, interactive scene control, and language grounding performance through extensive experiments. Project page: https://livescenes.github.io. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_16038 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Rendering and Control Qu, Delin Chen, Qizhi Zhang, Pingrui Gao, Xianqiang Li, Junzhe Zhao, Bin Wang, Dong Li, Xuelong Computer Vision and Pattern Recognition This paper scales object-level reconstruction to complex scenes, advancing interactive scene reconstruction. We introduce two datasets, OmniSim and InterReal, featuring 28 scenes with multiple interactive objects. To tackle the challenge of inaccurate interactive motion recovery in complex scenes, we propose LiveScene, a scene-level language-embedded interactive radiance field that efficiently reconstructs and controls multiple objects. By decomposing the interactive scene into local deformable fields, LiveScene enables separate reconstruction of individual object motions, reducing memory consumption. Additionally, our interaction-aware language embedding localizes individual interactive objects, allowing for arbitrary control using natural language. Our approach demonstrates significant superiority in novel view synthesis, interactive scene control, and language grounding performance through extensive experiments. Project page: https://livescenes.github.io. |
| title | LiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Rendering and Control |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.16038 |