CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objects
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909529428459520 |
|---|---|
| author | Heppert, Nick Irshad, Muhammad Zubair Zakharov, Sergey Liu, Katherine Ambrus, Rares Andrei Bohg, Jeannette Valada, Abhinav Kollar, Thomas |
| author_facet | Heppert, Nick Irshad, Muhammad Zubair Zakharov, Sergey Liu, Katherine Ambrus, Rares Andrei Bohg, Jeannette Valada, Abhinav Kollar, Thomas |
| contents | We present CARTO, a novel approach for reconstructing multiple articulated objects from a single stereo RGB observation. We use implicit object-centric representations and learn a single geometry and articulation decoder for multiple object categories. Despite training on multiple categories, our decoder achieves a comparable reconstruction accuracy to methods that train bespoke decoders separately for each category. Combined with our stereo image encoder we infer the 3D shape, 6D pose, size, joint type, and the joint state of multiple unknown objects in a single forward pass. Our method achieves a 20.4% absolute improvement in mAP 3D IOU50 for novel instances when compared to a two-stage pipeline. Inference time is fast and can run on a NVIDIA TITAN XP GPU at 1 HZ for eight or less objects present. While only trained on simulated data, CARTO transfers to real-world object instances. Code and evaluation data is available at: http://carto.cs.uni-freiburg.de |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_15782 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objects Heppert, Nick Irshad, Muhammad Zubair Zakharov, Sergey Liu, Katherine Ambrus, Rares Andrei Bohg, Jeannette Valada, Abhinav Kollar, Thomas Computer Vision and Pattern Recognition Robotics We present CARTO, a novel approach for reconstructing multiple articulated objects from a single stereo RGB observation. We use implicit object-centric representations and learn a single geometry and articulation decoder for multiple object categories. Despite training on multiple categories, our decoder achieves a comparable reconstruction accuracy to methods that train bespoke decoders separately for each category. Combined with our stereo image encoder we infer the 3D shape, 6D pose, size, joint type, and the joint state of multiple unknown objects in a single forward pass. Our method achieves a 20.4% absolute improvement in mAP 3D IOU50 for novel instances when compared to a two-stage pipeline. Inference time is fast and can run on a NVIDIA TITAN XP GPU at 1 HZ for eight or less objects present. While only trained on simulated data, CARTO transfers to real-world object instances. Code and evaluation data is available at: http://carto.cs.uni-freiburg.de |
| title | CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objects |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2303.15782 |