HIPPo: Harnessing Image-to-3D Priors for Model-free Zero-shot 6D Pose Estimation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929717172502528 |
|---|---|
| author | Liu, Yibo Jiang, Zhaodong Xu, Binbin Wu, Guile Ren, Yuan Cao, Tongtong Liu, Bingbing Yang, Rui Heng Rasouli, Amir Shan, Jinjun |
| author_facet | Liu, Yibo Jiang, Zhaodong Xu, Binbin Wu, Guile Ren, Yuan Cao, Tongtong Liu, Bingbing Yang, Rui Heng Rasouli, Amir Shan, Jinjun |
| contents | This work focuses on model-free zero-shot 6D object pose estimation for robotics applications. While existing methods can estimate the precise 6D pose of objects, they heavily rely on curated CAD models or reference images, the preparation of which is a time-consuming and labor-intensive process. Moreover, in real-world scenarios, 3D models or reference images may not be available in advance and instant robot reaction is desired. In this work, we propose a novel framework named HIPPo, which eliminates the need for curated CAD models and reference images by harnessing image-to-3D priors from Diffusion Models, enabling model-free zero-shot 6D pose estimation. Specifically, we construct HIPPo Dreamer, a rapid image-to-mesh model built on a multiview Diffusion Model and a 3D reconstruction foundation model. Our HIPPo Dreamer can generate a 3D mesh of any unseen objects from a single glance in just a few seconds. Then, as more observations are acquired, we propose to continuously refine the diffusion prior mesh model by joint optimization of object geometry and appearance. This is achieved by a measurement-guided scheme that gradually replaces the plausible diffusion priors with more reliable online observations. Consequently, HIPPo can instantly estimate and track the 6D pose of a novel object and maintain a complete mesh for immediate robotic applications. Thorough experiments on various benchmarks show that HIPPo outperforms state-of-the-art methods in 6D object pose estimation when prior reference images are limited. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_10606 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HIPPo: Harnessing Image-to-3D Priors for Model-free Zero-shot 6D Pose Estimation Liu, Yibo Jiang, Zhaodong Xu, Binbin Wu, Guile Ren, Yuan Cao, Tongtong Liu, Bingbing Yang, Rui Heng Rasouli, Amir Shan, Jinjun Computer Vision and Pattern Recognition Robotics This work focuses on model-free zero-shot 6D object pose estimation for robotics applications. While existing methods can estimate the precise 6D pose of objects, they heavily rely on curated CAD models or reference images, the preparation of which is a time-consuming and labor-intensive process. Moreover, in real-world scenarios, 3D models or reference images may not be available in advance and instant robot reaction is desired. In this work, we propose a novel framework named HIPPo, which eliminates the need for curated CAD models and reference images by harnessing image-to-3D priors from Diffusion Models, enabling model-free zero-shot 6D pose estimation. Specifically, we construct HIPPo Dreamer, a rapid image-to-mesh model built on a multiview Diffusion Model and a 3D reconstruction foundation model. Our HIPPo Dreamer can generate a 3D mesh of any unseen objects from a single glance in just a few seconds. Then, as more observations are acquired, we propose to continuously refine the diffusion prior mesh model by joint optimization of object geometry and appearance. This is achieved by a measurement-guided scheme that gradually replaces the plausible diffusion priors with more reliable online observations. Consequently, HIPPo can instantly estimate and track the 6D pose of a novel object and maintain a complete mesh for immediate robotic applications. Thorough experiments on various benchmarks show that HIPPo outperforms state-of-the-art methods in 6D object pose estimation when prior reference images are limited. |
| title | HIPPo: Harnessing Image-to-3D Priors for Model-free Zero-shot 6D Pose Estimation |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2502.10606 |