From Scan to Action: Leveraging Realistic Scans for Embodied Scene Understanding
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911073031946240 |
|---|---|
| author | Halacheva, Anna-Maria Zaech, Jan-Nico Dey, Sombit Van Gool, Luc Paudel, Danda Pani |
| author_facet | Halacheva, Anna-Maria Zaech, Jan-Nico Dey, Sombit Van Gool, Luc Paudel, Danda Pani |
| contents | Real-world 3D scene-level scans offer realism and can enable better real-world generalizability for downstream applications. However, challenges such as data volume, diverse annotation formats, and tool compatibility limit their use. This paper demonstrates a methodology to effectively leverage these scans and their annotations. We propose a unified annotation integration using USD, with application-specific USD flavors. We identify challenges in utilizing holistic real-world scan datasets and present mitigation strategies. The efficacy of our approach is demonstrated through two downstream applications: LLM-based scene editing, enabling effective LLM understanding and adaptation of the data (80% success), and robotic simulation, achieving an 87% success rate in policy learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_17585 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Scan to Action: Leveraging Realistic Scans for Embodied Scene Understanding Halacheva, Anna-Maria Zaech, Jan-Nico Dey, Sombit Van Gool, Luc Paudel, Danda Pani Computer Vision and Pattern Recognition Robotics Real-world 3D scene-level scans offer realism and can enable better real-world generalizability for downstream applications. However, challenges such as data volume, diverse annotation formats, and tool compatibility limit their use. This paper demonstrates a methodology to effectively leverage these scans and their annotations. We propose a unified annotation integration using USD, with application-specific USD flavors. We identify challenges in utilizing holistic real-world scan datasets and present mitigation strategies. The efficacy of our approach is demonstrated through two downstream applications: LLM-based scene editing, enabling effective LLM understanding and adaptation of the data (80% success), and robotic simulation, achieving an 87% success rate in policy learning. |
| title | From Scan to Action: Leveraging Realistic Scans for Embodied Scene Understanding |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2507.17585 |