Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.12779 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915200603521024 |
|---|---|
| author | Wang, Haoxiao Zhou, Kaichen Gu, Binrui Feng, Zhiyuan Wang, Weijie Sun, Peilin Xiao, Yicheng Zhang, Jianhua Dong, Hao |
| author_facet | Wang, Haoxiao Zhou, Kaichen Gu, Binrui Feng, Zhiyuan Wang, Weijie Sun, Peilin Xiao, Yicheng Zhang, Jianhua Dong, Hao |
| contents | Manipulating transparent objects presents significant challenges due to the complexities introduced by their reflection and refraction properties, which considerably hinder the accurate estimation of their 3D shapes. To address these challenges, we propose a single-view RGB-D-based depth completion framework, TransDiff, that leverages the Denoising Diffusion Probabilistic Models(DDPM) to achieve material-agnostic object grasping in desktop. Specifically, we leverage features extracted from RGB images, including semantic segmentation, edge maps, and normal maps, to condition the depth map generation process. Our method learns an iterative denoising process that transforms a random depth distribution into a depth map, guided by initially refined depth information, ensuring more accurate depth estimation in scenarios involving transparent objects. Additionally, we propose a novel training method to better align the noisy depth and RGB image features, which are used as conditions to refine depth estimation step by step. Finally, we utilized an improved inference process to accelerate the denoising procedure. Through comprehensive experimental validation, we demonstrate that our method significantly outperforms the baselines in both synthetic and real-world benchmarks with acceptable inference time. The demo of our method can be found on https://wang-haoxiao.github.io/TransDiff/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_12779 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TransDiff: Diffusion-Based Method for Manipulating Transparent Objects Using a Single RGB-D Image Wang, Haoxiao Zhou, Kaichen Gu, Binrui Feng, Zhiyuan Wang, Weijie Sun, Peilin Xiao, Yicheng Zhang, Jianhua Dong, Hao Computer Vision and Pattern Recognition Manipulating transparent objects presents significant challenges due to the complexities introduced by their reflection and refraction properties, which considerably hinder the accurate estimation of their 3D shapes. To address these challenges, we propose a single-view RGB-D-based depth completion framework, TransDiff, that leverages the Denoising Diffusion Probabilistic Models(DDPM) to achieve material-agnostic object grasping in desktop. Specifically, we leverage features extracted from RGB images, including semantic segmentation, edge maps, and normal maps, to condition the depth map generation process. Our method learns an iterative denoising process that transforms a random depth distribution into a depth map, guided by initially refined depth information, ensuring more accurate depth estimation in scenarios involving transparent objects. Additionally, we propose a novel training method to better align the noisy depth and RGB image features, which are used as conditions to refine depth estimation step by step. Finally, we utilized an improved inference process to accelerate the denoising procedure. Through comprehensive experimental validation, we demonstrate that our method significantly outperforms the baselines in both synthetic and real-world benchmarks with acceptable inference time. The demo of our method can be found on https://wang-haoxiao.github.io/TransDiff/ |
| title | TransDiff: Diffusion-Based Method for Manipulating Transparent Objects Using a Single RGB-D Image |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.12779 |