Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913634293121024 |
|---|---|
| author | Longhini, Alberta Büsching, Marcel Duisterhof, Bardienus P. Lundell, Jens Ichnowski, Jeffrey Björkman, Mårten Kragic, Danica |
| author_facet | Longhini, Alberta Büsching, Marcel Duisterhof, Bardienus P. Lundell, Jens Ichnowski, Jeffrey Björkman, Mårten Kragic, Danica |
| contents | We introduce Cloth-Splatting, a method for estimating 3D states of cloth from RGB images through a prediction-update framework. Cloth-Splatting leverages an action-conditioned dynamics model for predicting future states and uses 3D Gaussian Splatting to update the predicted states. Our key insight is that coupling a 3D mesh-based representation with Gaussian Splatting allows us to define a differentiable map between the cloth state space and the image space. This enables the use of gradient-based optimization techniques to refine inaccurate state estimates using only RGB supervision. Our experiments demonstrate that Cloth-Splatting not only improves state estimation accuracy over current baselines but also reduces convergence time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_01715 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision Longhini, Alberta Büsching, Marcel Duisterhof, Bardienus P. Lundell, Jens Ichnowski, Jeffrey Björkman, Mårten Kragic, Danica Computer Vision and Pattern Recognition Robotics We introduce Cloth-Splatting, a method for estimating 3D states of cloth from RGB images through a prediction-update framework. Cloth-Splatting leverages an action-conditioned dynamics model for predicting future states and uses 3D Gaussian Splatting to update the predicted states. Our key insight is that coupling a 3D mesh-based representation with Gaussian Splatting allows us to define a differentiable map between the cloth state space and the image space. This enables the use of gradient-based optimization techniques to refine inaccurate state estimates using only RGB supervision. Our experiments demonstrate that Cloth-Splatting not only improves state estimation accuracy over current baselines but also reduces convergence time. |
| title | Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2501.01715 |