RayFronts: Open-Set Semantic Ray Frontiers for Online Scene Understanding and Exploration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913786355515392 |
|---|---|
| author | Alama, Omar Bhattacharya, Avigyan He, Haoyang Kim, Seungchan Qiu, Yuheng Wang, Wenshan Ho, Cherie Keetha, Nikhil Scherer, Sebastian |
| author_facet | Alama, Omar Bhattacharya, Avigyan He, Haoyang Kim, Seungchan Qiu, Yuheng Wang, Wenshan Ho, Cherie Keetha, Nikhil Scherer, Sebastian |
| contents | Open-set semantic mapping is crucial for open-world robots. Current mapping approaches either are limited by the depth range or only map beyond-range entities in constrained settings, where overall they fail to combine within-range and beyond-range observations. Furthermore, these methods make a trade-off between fine-grained semantics and efficiency. We introduce RayFronts, a unified representation that enables both dense and beyond-range efficient semantic mapping. RayFronts encodes task-agnostic open-set semantics to both in-range voxels and beyond-range rays encoded at map boundaries, empowering the robot to reduce search volumes significantly and make informed decisions both within & beyond sensory range, while running at 8.84 Hz on an Orin AGX. Benchmarking the within-range semantics shows that RayFronts's fine-grained image encoding provides 1.34x zero-shot 3D semantic segmentation performance while improving throughput by 16.5x. Traditionally, online mapping performance is entangled with other system components, complicating evaluation. We propose a planner-agnostic evaluation framework that captures the utility for online beyond-range search and exploration, and show RayFronts reduces search volume 2.2x more efficiently than the closest online baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_06994 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RayFronts: Open-Set Semantic Ray Frontiers for Online Scene Understanding and Exploration Alama, Omar Bhattacharya, Avigyan He, Haoyang Kim, Seungchan Qiu, Yuheng Wang, Wenshan Ho, Cherie Keetha, Nikhil Scherer, Sebastian Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Open-set semantic mapping is crucial for open-world robots. Current mapping approaches either are limited by the depth range or only map beyond-range entities in constrained settings, where overall they fail to combine within-range and beyond-range observations. Furthermore, these methods make a trade-off between fine-grained semantics and efficiency. We introduce RayFronts, a unified representation that enables both dense and beyond-range efficient semantic mapping. RayFronts encodes task-agnostic open-set semantics to both in-range voxels and beyond-range rays encoded at map boundaries, empowering the robot to reduce search volumes significantly and make informed decisions both within & beyond sensory range, while running at 8.84 Hz on an Orin AGX. Benchmarking the within-range semantics shows that RayFronts's fine-grained image encoding provides 1.34x zero-shot 3D semantic segmentation performance while improving throughput by 16.5x. Traditionally, online mapping performance is entangled with other system components, complicating evaluation. We propose a planner-agnostic evaluation framework that captures the utility for online beyond-range search and exploration, and show RayFronts reduces search volume 2.2x more efficiently than the closest online baselines. |
| title | RayFronts: Open-Set Semantic Ray Frontiers for Online Scene Understanding and Exploration |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2504.06994 |