Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2308.14666 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914748544581632 |
|---|---|
| author | Mason, Justice Allen-Blanchette, Christine Zolman, Nicholas Davison, Elizabeth Leonard, Naomi Ehrich |
| author_facet | Mason, Justice Allen-Blanchette, Christine Zolman, Nicholas Davison, Elizabeth Leonard, Naomi Ehrich |
| contents | In many real-world settings, image observations of freely rotating 3D rigid bodies may be available when low-dimensional measurements are not. However, the high-dimensionality of image data precludes the use of classical estimation techniques to learn the dynamics. The usefulness of standard deep learning methods is also limited, because an image of a rigid body reveals nothing about the distribution of mass inside the body, which, together with initial angular velocity, is what determines how the body will rotate. We present a physics-based neural network model to estimate and predict 3D rotational dynamics from image sequences. We achieve this using a multi-stage prediction pipeline that maps individual images to a latent representation homeomorphic to $\mathbf{SO}(3)$, computes angular velocities from latent pairs, and predicts future latent states using the Hamiltonian equations of motion. We demonstrate the efficacy of our approach on new rotating rigid-body datasets of sequences of synthetic images of rotating objects, including cubes, prisms and satellites, with unknown uniform and non-uniform mass distributions. Our model outperforms competing baselines on our datasets, producing better qualitative predictions and reducing the error observed for the state-of-the-art Hamiltonian Generative Network by a factor of 2. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_14666 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Learning to Predict 3D Rotational Dynamics from Images of a Rigid Body with Unknown Mass Distribution Mason, Justice Allen-Blanchette, Christine Zolman, Nicholas Davison, Elizabeth Leonard, Naomi Ehrich Computer Vision and Pattern Recognition Computational Engineering, Finance, and Science Machine Learning In many real-world settings, image observations of freely rotating 3D rigid bodies may be available when low-dimensional measurements are not. However, the high-dimensionality of image data precludes the use of classical estimation techniques to learn the dynamics. The usefulness of standard deep learning methods is also limited, because an image of a rigid body reveals nothing about the distribution of mass inside the body, which, together with initial angular velocity, is what determines how the body will rotate. We present a physics-based neural network model to estimate and predict 3D rotational dynamics from image sequences. We achieve this using a multi-stage prediction pipeline that maps individual images to a latent representation homeomorphic to $\mathbf{SO}(3)$, computes angular velocities from latent pairs, and predicts future latent states using the Hamiltonian equations of motion. We demonstrate the efficacy of our approach on new rotating rigid-body datasets of sequences of synthetic images of rotating objects, including cubes, prisms and satellites, with unknown uniform and non-uniform mass distributions. Our model outperforms competing baselines on our datasets, producing better qualitative predictions and reducing the error observed for the state-of-the-art Hamiltonian Generative Network by a factor of 2. |
| title | Learning to Predict 3D Rotational Dynamics from Images of a Rigid Body with Unknown Mass Distribution |
| topic | Computer Vision and Pattern Recognition Computational Engineering, Finance, and Science Machine Learning |
| url | https://arxiv.org/abs/2308.14666 |