SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911811081601024 |
|---|---|
| author | Zhai, Guangyao Cai, Xiaoni Huang, Dianye Di, Yan Manhardt, Fabian Tombari, Federico Navab, Nassir Busam, Benjamin |
| author_facet | Zhai, Guangyao Cai, Xiaoni Huang, Dianye Di, Yan Manhardt, Fabian Tombari, Federico Navab, Nassir Busam, Benjamin |
| contents | Object rearrangement is pivotal in robotic-environment interactions, representing a significant capability in embodied AI. In this paper, we present SG-Bot, a novel rearrangement framework that utilizes a coarse-to-fine scheme with a scene graph as the scene representation. Unlike previous methods that rely on either known goal priors or zero-shot large models, SG-Bot exemplifies lightweight, real-time, and user-controllable characteristics, seamlessly blending the consideration of commonsense knowledge with automatic generation capabilities. SG-Bot employs a three-fold procedure--observation, imagination, and execution--to adeptly address the task. Initially, objects are discerned and extracted from a cluttered scene during the observation. These objects are first coarsely organized and depicted within a scene graph, guided by either commonsense or user-defined criteria. Then, this scene graph subsequently informs a generative model, which forms a fine-grained goal scene considering the shape information from the initial scene and object semantics. Finally, for execution, the initial and envisioned goal scenes are matched to formulate robotic action policies. Experimental results demonstrate that SG-Bot outperforms competitors by a large margin. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_12188 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphs Zhai, Guangyao Cai, Xiaoni Huang, Dianye Di, Yan Manhardt, Fabian Tombari, Federico Navab, Nassir Busam, Benjamin Robotics Computer Vision and Pattern Recognition Object rearrangement is pivotal in robotic-environment interactions, representing a significant capability in embodied AI. In this paper, we present SG-Bot, a novel rearrangement framework that utilizes a coarse-to-fine scheme with a scene graph as the scene representation. Unlike previous methods that rely on either known goal priors or zero-shot large models, SG-Bot exemplifies lightweight, real-time, and user-controllable characteristics, seamlessly blending the consideration of commonsense knowledge with automatic generation capabilities. SG-Bot employs a three-fold procedure--observation, imagination, and execution--to adeptly address the task. Initially, objects are discerned and extracted from a cluttered scene during the observation. These objects are first coarsely organized and depicted within a scene graph, guided by either commonsense or user-defined criteria. Then, this scene graph subsequently informs a generative model, which forms a fine-grained goal scene considering the shape information from the initial scene and object semantics. Finally, for execution, the initial and envisioned goal scenes are matched to formulate robotic action policies. Experimental results demonstrate that SG-Bot outperforms competitors by a large margin. |
| title | SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphs |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2309.12188 |