Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.10494 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916160485720064 |
|---|---|
| author | Rashid, Adam Kim, Chung Min Kerr, Justin Fu, Letian Hari, Kush Ahmad, Ayah Chen, Kaiyuan Huang, Huang Gualtieri, Marcus Wang, Michael Juette, Christian Tian, Nan Ren, Liu Goldberg, Ken |
| author_facet | Rashid, Adam Kim, Chung Min Kerr, Justin Fu, Letian Hari, Kush Ahmad, Ayah Chen, Kaiyuan Huang, Huang Gualtieri, Marcus Wang, Michael Juette, Christian Tian, Nan Ren, Liu Goldberg, Ken |
| contents | Inventory monitoring in homes, factories, and retail stores relies on maintaining data despite objects being swapped, added, removed, or moved. We introduce Lifelong LERF, a method that allows a mobile robot with minimal compute to jointly optimize a dense language and geometric representation of its surroundings. Lifelong LERF maintains this representation over time by detecting semantic changes and selectively updating these regions of the environment, avoiding the need to exhaustively remap. Human users can query inventory by providing natural language queries and receiving a 3D heatmap of potential object locations. To manage the computational load, we use Fog-ROS2, a cloud robotics platform, to offload resource-intensive tasks. Lifelong LERF obtains poses from a monocular RGBD SLAM backend, and uses these poses to progressively optimize a Language Embedded Radiance Field (LERF) for semantic monitoring. Experiments with 3-5 objects arranged on a tabletop and a Turtlebot with a RealSense camera suggest that Lifelong LERF can persistently adapt to changes in objects with up to 91% accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_10494 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Lifelong LERF: Local 3D Semantic Inventory Monitoring Using FogROS2 Rashid, Adam Kim, Chung Min Kerr, Justin Fu, Letian Hari, Kush Ahmad, Ayah Chen, Kaiyuan Huang, Huang Gualtieri, Marcus Wang, Michael Juette, Christian Tian, Nan Ren, Liu Goldberg, Ken Robotics Inventory monitoring in homes, factories, and retail stores relies on maintaining data despite objects being swapped, added, removed, or moved. We introduce Lifelong LERF, a method that allows a mobile robot with minimal compute to jointly optimize a dense language and geometric representation of its surroundings. Lifelong LERF maintains this representation over time by detecting semantic changes and selectively updating these regions of the environment, avoiding the need to exhaustively remap. Human users can query inventory by providing natural language queries and receiving a 3D heatmap of potential object locations. To manage the computational load, we use Fog-ROS2, a cloud robotics platform, to offload resource-intensive tasks. Lifelong LERF obtains poses from a monocular RGBD SLAM backend, and uses these poses to progressively optimize a Language Embedded Radiance Field (LERF) for semantic monitoring. Experiments with 3-5 objects arranged on a tabletop and a Turtlebot with a RealSense camera suggest that Lifelong LERF can persistently adapt to changes in objects with up to 91% accuracy. |
| title | Lifelong LERF: Local 3D Semantic Inventory Monitoring Using FogROS2 |
| topic | Robotics |
| url | https://arxiv.org/abs/2403.10494 |