Learning to Estimate Single-View Volumetric Flow Motions without 3D Supervision
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909542835552256 |
|---|---|
| author | Franz, Aleksandra Solenthaler, Barbara Thuerey, Nils |
| author_facet | Franz, Aleksandra Solenthaler, Barbara Thuerey, Nils |
| contents | We address the challenging problem of jointly inferring the 3D flow and volumetric densities moving in a fluid from a monocular input video with a deep neural network. Despite the complexity of this task, we show that it is possible to train the corresponding networks without requiring any 3D ground truth for training. In the absence of ground truth data we can train our model with observations from real-world capture setups instead of relying on synthetic reconstructions. We make this unsupervised training approach possible by first generating an initial prototype volume which is then moved and transported over time without the need for volumetric supervision. Our approach relies purely on image-based losses, an adversarial discriminator network, and regularization. Our method can estimate long-term sequences in a stable manner, while achieving closely matching targets for inputs such as rising smoke plumes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_14470 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Learning to Estimate Single-View Volumetric Flow Motions without 3D Supervision Franz, Aleksandra Solenthaler, Barbara Thuerey, Nils Computer Vision and Pattern Recognition Machine Learning Fluid Dynamics We address the challenging problem of jointly inferring the 3D flow and volumetric densities moving in a fluid from a monocular input video with a deep neural network. Despite the complexity of this task, we show that it is possible to train the corresponding networks without requiring any 3D ground truth for training. In the absence of ground truth data we can train our model with observations from real-world capture setups instead of relying on synthetic reconstructions. We make this unsupervised training approach possible by first generating an initial prototype volume which is then moved and transported over time without the need for volumetric supervision. Our approach relies purely on image-based losses, an adversarial discriminator network, and regularization. Our method can estimate long-term sequences in a stable manner, while achieving closely matching targets for inputs such as rising smoke plumes. |
| title | Learning to Estimate Single-View Volumetric Flow Motions without 3D Supervision |
| topic | Computer Vision and Pattern Recognition Machine Learning Fluid Dynamics |
| url | https://arxiv.org/abs/2302.14470 |