Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.13848 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910751429492736 |
|---|---|
| author | Zhang, Jinrui Zhang, Deyu Long, Tingting Chen, Wenxin Ren, Ju Liu, Yunxin Zhao, Yudong Zhang, Yaoxue Lee, Youngki |
| author_facet | Zhang, Jinrui Zhang, Deyu Long, Tingting Chen, Wenxin Ren, Ju Liu, Yunxin Zhao, Yudong Zhang, Yaoxue Lee, Youngki |
| contents | We present MobiFuse, a high-precision depth perception system on mobile devices that combines dual RGB and Time-of-Flight (ToF) cameras. To achieve this, we leverage physical principles from various environmental factors to propose the Depth Error Indication (DEI) modality, characterizing the depth error of ToF and stereo-matching. Furthermore, we employ a progressive fusion strategy, merging geometric features from ToF and stereo depth maps with depth error features from the DEI modality to create precise depth maps. Additionally, we create a new ToF-Stereo depth dataset, RealToF, to train and validate our model. Our experiments demonstrate that MobiFuse excels over baselines by significantly reducing depth measurement errors by up to 77.7%. It also showcases strong generalization across diverse datasets and proves effectiveness in two downstream tasks: 3D reconstruction and 3D segmentation. The demo video of MobiFuse in real-life scenarios is available at the de-identified YouTube link(https://youtu.be/jy-Sp7T1LVs). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_13848 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MobiFuse: A High-Precision On-device Depth Perception System with Multi-Data Fusion Zhang, Jinrui Zhang, Deyu Long, Tingting Chen, Wenxin Ren, Ju Liu, Yunxin Zhao, Yudong Zhang, Yaoxue Lee, Youngki Computer Vision and Pattern Recognition We present MobiFuse, a high-precision depth perception system on mobile devices that combines dual RGB and Time-of-Flight (ToF) cameras. To achieve this, we leverage physical principles from various environmental factors to propose the Depth Error Indication (DEI) modality, characterizing the depth error of ToF and stereo-matching. Furthermore, we employ a progressive fusion strategy, merging geometric features from ToF and stereo depth maps with depth error features from the DEI modality to create precise depth maps. Additionally, we create a new ToF-Stereo depth dataset, RealToF, to train and validate our model. Our experiments demonstrate that MobiFuse excels over baselines by significantly reducing depth measurement errors by up to 77.7%. It also showcases strong generalization across diverse datasets and proves effectiveness in two downstream tasks: 3D reconstruction and 3D segmentation. The demo video of MobiFuse in real-life scenarios is available at the de-identified YouTube link(https://youtu.be/jy-Sp7T1LVs). |
| title | MobiFuse: A High-Precision On-device Depth Perception System with Multi-Data Fusion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.13848 |