Understanding Particles From Video: Property Estimation of Granular Materials via Visuo-Haptic Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915045654396928 |
|---|---|
| author | Zhang, Zeqing Zheng, Guangze Ji, Xuebo Chen, Guanqi Jia, Ruixing Chen, Wentao Chen, Guanhua Zhang, Liangjun Pan, Jia |
| author_facet | Zhang, Zeqing Zheng, Guangze Ji, Xuebo Chen, Guanqi Jia, Ruixing Chen, Wentao Chen, Guanhua Zhang, Liangjun Pan, Jia |
| contents | Granular materials (GMs) are ubiquitous in daily life. Understanding their properties is also important, especially in agriculture and industry. However, existing works require dedicated measurement equipment and also need large human efforts to handle a large number of particles. In this paper, we introduce a method for estimating the relative values of particle size and density from the video of the interaction with GMs. It is trained on a visuo-haptic learning framework inspired by a contact model, which reveals the strong correlation between GM properties and the visual-haptic data during the probe-dragging in the GMs. After training, the network can map the visual modality well to the haptic signal and implicitly characterize the relative distribution of particle properties in its latent embeddings, as interpreted in that contact model. Therefore, we can analyze GM properties using the trained encoder, and only visual information is needed without extra sensory modalities and human efforts for labeling. The presented GM property estimator has been extensively validated via comparison and ablation experiments. The generalization capability has also been evaluated and a real-world application on the beach is also demonstrated. Experiment videos are available at \url{https://sites.google.com/view/gmwork/vhlearning} . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_02119 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Understanding Particles From Video: Property Estimation of Granular Materials via Visuo-Haptic Learning Zhang, Zeqing Zheng, Guangze Ji, Xuebo Chen, Guanqi Jia, Ruixing Chen, Wentao Chen, Guanhua Zhang, Liangjun Pan, Jia Computer Vision and Pattern Recognition Machine Learning Robotics Granular materials (GMs) are ubiquitous in daily life. Understanding their properties is also important, especially in agriculture and industry. However, existing works require dedicated measurement equipment and also need large human efforts to handle a large number of particles. In this paper, we introduce a method for estimating the relative values of particle size and density from the video of the interaction with GMs. It is trained on a visuo-haptic learning framework inspired by a contact model, which reveals the strong correlation between GM properties and the visual-haptic data during the probe-dragging in the GMs. After training, the network can map the visual modality well to the haptic signal and implicitly characterize the relative distribution of particle properties in its latent embeddings, as interpreted in that contact model. Therefore, we can analyze GM properties using the trained encoder, and only visual information is needed without extra sensory modalities and human efforts for labeling. The presented GM property estimator has been extensively validated via comparison and ablation experiments. The generalization capability has also been evaluated and a real-world application on the beach is also demonstrated. Experiment videos are available at \url{https://sites.google.com/view/gmwork/vhlearning} . |
| title | Understanding Particles From Video: Property Estimation of Granular Materials via Visuo-Haptic Learning |
| topic | Computer Vision and Pattern Recognition Machine Learning Robotics |
| url | https://arxiv.org/abs/2412.02119 |