Seeing the Wind from a Falling Leaf
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917114421444608 |
|---|---|
| author | Gao, Zhiyuan Mao, Jiageng Yu, Hong-Xing Lou, Haozhe Jia, Emily Yue-Ting Barbic, Jernej Wu, Jiajun Wang, Yue |
| author_facet | Gao, Zhiyuan Mao, Jiageng Yu, Hong-Xing Lou, Haozhe Jia, Emily Yue-Ting Barbic, Jernej Wu, Jiajun Wang, Yue |
| contents | A longstanding goal in computer vision is to model motions from videos, while the representations behind motions, i.e. the invisible physical interactions that cause objects to deform and move, remain largely unexplored. In this paper, we study how to recover the invisible forces from visual observations, e.g., estimating the wind field by observing a leaf falling to the ground. Our key innovation is an end-to-end differentiable inverse graphics framework, which jointly models object geometry, physical properties, and interactions directly from videos. Through backpropagation, our approach enables the recovery of force representations from object motions. We validate our method on both synthetic and real-world scenarios, and the results demonstrate its ability to infer plausible force fields from videos. Furthermore, we show the potential applications of our approach, including physics-based video generation and editing. We hope our approach sheds light on understanding and modeling the physical process behind pixels, bridging the gap between vision and physics. Please check more video results in our \href{https://chaoren2357.github.io/seeingthewind/}{project page}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_00762 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Seeing the Wind from a Falling Leaf Gao, Zhiyuan Mao, Jiageng Yu, Hong-Xing Lou, Haozhe Jia, Emily Yue-Ting Barbic, Jernej Wu, Jiajun Wang, Yue Computer Vision and Pattern Recognition A longstanding goal in computer vision is to model motions from videos, while the representations behind motions, i.e. the invisible physical interactions that cause objects to deform and move, remain largely unexplored. In this paper, we study how to recover the invisible forces from visual observations, e.g., estimating the wind field by observing a leaf falling to the ground. Our key innovation is an end-to-end differentiable inverse graphics framework, which jointly models object geometry, physical properties, and interactions directly from videos. Through backpropagation, our approach enables the recovery of force representations from object motions. We validate our method on both synthetic and real-world scenarios, and the results demonstrate its ability to infer plausible force fields from videos. Furthermore, we show the potential applications of our approach, including physics-based video generation and editing. We hope our approach sheds light on understanding and modeling the physical process behind pixels, bridging the gap between vision and physics. Please check more video results in our \href{https://chaoren2357.github.io/seeingthewind/}{project page}. |
| title | Seeing the Wind from a Falling Leaf |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.00762 |