Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2312.08879 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913466281885696 |
|---|---|
| author | Vacek, Patrik Hurych, David Zimmermann, Karel Perez, Patrick Svoboda, Tomas |
| author_facet | Vacek, Patrik Hurych, David Zimmermann, Karel Perez, Patrick Svoboda, Tomas |
| contents | Learning without supervision how to predict 3D scene flows from point clouds is essential to many perception systems. We propose a novel learning framework for this task which improves the necessary regularization. Relying on the assumption that scene elements are mostly rigid, current smoothness losses are built on the definition of "rigid clusters" in the input point clouds. The definition of these clusters is challenging and has a significant impact on the quality of predicted flows. We introduce two new consistency losses that enlarge clusters while preventing them from spreading over distinct objects. In particular, we enforce \emph{temporal} consistency with a forward-backward cyclic loss and \emph{spatial} consistency by considering surface orientation similarity in addition to spatial proximity. The proposed losses are model-independent and can thus be used in a plug-and-play fashion to significantly improve the performance of existing models, as demonstrated on two most widely used architectures. We also showcase the effectiveness and generalization capability of our framework on four standard sensor-unique driving datasets, achieving state-of-the-art performance in 3D scene flow estimation. Our codes are available on https://github.com/ctu-vras/sac-flow. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_08879 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Regularizing Self-supervised 3D Scene Flows with Surface Awareness and Cyclic Consistency Vacek, Patrik Hurych, David Zimmermann, Karel Perez, Patrick Svoboda, Tomas Computer Vision and Pattern Recognition Learning without supervision how to predict 3D scene flows from point clouds is essential to many perception systems. We propose a novel learning framework for this task which improves the necessary regularization. Relying on the assumption that scene elements are mostly rigid, current smoothness losses are built on the definition of "rigid clusters" in the input point clouds. The definition of these clusters is challenging and has a significant impact on the quality of predicted flows. We introduce two new consistency losses that enlarge clusters while preventing them from spreading over distinct objects. In particular, we enforce \emph{temporal} consistency with a forward-backward cyclic loss and \emph{spatial} consistency by considering surface orientation similarity in addition to spatial proximity. The proposed losses are model-independent and can thus be used in a plug-and-play fashion to significantly improve the performance of existing models, as demonstrated on two most widely used architectures. We also showcase the effectiveness and generalization capability of our framework on four standard sensor-unique driving datasets, achieving state-of-the-art performance in 3D scene flow estimation. Our codes are available on https://github.com/ctu-vras/sac-flow. |
| title | Regularizing Self-supervised 3D Scene Flows with Surface Awareness and Cyclic Consistency |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.08879 |