The Sound of Water: Inferring Physical Properties from Pouring Liquids
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929673376628736 |
|---|---|
| author | Bagad, Piyush Tapaswi, Makarand Snoek, Cees G. M. Zisserman, Andrew |
| author_facet | Bagad, Piyush Tapaswi, Makarand Snoek, Cees G. M. Zisserman, Andrew |
| contents | We study the connection between audio-visual observations and the underlying physics of a mundane yet intriguing everyday activity: pouring liquids. Given only the sound of liquid pouring into a container, our objective is to automatically infer physical properties such as the liquid level, the shape and size of the container, the pouring rate and the time to fill. To this end, we: (i) show in theory that these properties can be determined from the fundamental frequency (pitch); (ii) train a pitch detection model with supervision from simulated data and visual data with a physics-inspired objective; (iii) introduce a new large dataset of real pouring videos for a systematic study; (iv) show that the trained model can indeed infer these physical properties for real data; and finally, (v) we demonstrate strong generalization to various container shapes, other datasets, and in-the-wild YouTube videos. Our work presents a keen understanding of a narrow yet rich problem at the intersection of acoustics, physics, and learning. It opens up applications to enhance multisensory perception in robotic pouring. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11222 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Sound of Water: Inferring Physical Properties from Pouring Liquids Bagad, Piyush Tapaswi, Makarand Snoek, Cees G. M. Zisserman, Andrew Computer Vision and Pattern Recognition Multimedia Sound Audio and Speech Processing We study the connection between audio-visual observations and the underlying physics of a mundane yet intriguing everyday activity: pouring liquids. Given only the sound of liquid pouring into a container, our objective is to automatically infer physical properties such as the liquid level, the shape and size of the container, the pouring rate and the time to fill. To this end, we: (i) show in theory that these properties can be determined from the fundamental frequency (pitch); (ii) train a pitch detection model with supervision from simulated data and visual data with a physics-inspired objective; (iii) introduce a new large dataset of real pouring videos for a systematic study; (iv) show that the trained model can indeed infer these physical properties for real data; and finally, (v) we demonstrate strong generalization to various container shapes, other datasets, and in-the-wild YouTube videos. Our work presents a keen understanding of a narrow yet rich problem at the intersection of acoustics, physics, and learning. It opens up applications to enhance multisensory perception in robotic pouring. |
| title | The Sound of Water: Inferring Physical Properties from Pouring Liquids |
| topic | Computer Vision and Pattern Recognition Multimedia Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2411.11222 |