Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913657779126272 |
|---|---|
| author | Hamann, Friedhelm Wang, Ziyun Asmanis, Ioannis Chaney, Kenneth Gallego, Guillermo Daniilidis, Kostas |
| author_facet | Hamann, Friedhelm Wang, Ziyun Asmanis, Ioannis Chaney, Kenneth Gallego, Guillermo Daniilidis, Kostas |
| contents | Current optical flow and point-tracking methods rely heavily on synthetic datasets. Event cameras are novel vision sensors with advantages in challenging visual conditions, but state-of-the-art frame-based methods cannot be easily adapted to event data due to the limitations of current event simulators. We introduce a novel self-supervised loss combining the Contrast Maximization framework with a non-linear motion prior in the form of pixel-level trajectories and propose an efficient solution to solve the high-dimensional assignment problem between non-linear trajectories and events. Their effectiveness is demonstrated in two scenarios: In dense continuous-time motion estimation, our method improves the zero-shot performance of a synthetically trained model on the real-world dataset EVIMO2 by 29%. In optical flow estimation, our method elevates a simple UNet to achieve state-of-the-art performance among self-supervised methods on the DSEC optical flow benchmark. Our code is available at https://github.com/tub-rip/MotionPriorCMax. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_10802 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation Hamann, Friedhelm Wang, Ziyun Asmanis, Ioannis Chaney, Kenneth Gallego, Guillermo Daniilidis, Kostas Computer Vision and Pattern Recognition Machine Learning Robotics Current optical flow and point-tracking methods rely heavily on synthetic datasets. Event cameras are novel vision sensors with advantages in challenging visual conditions, but state-of-the-art frame-based methods cannot be easily adapted to event data due to the limitations of current event simulators. We introduce a novel self-supervised loss combining the Contrast Maximization framework with a non-linear motion prior in the form of pixel-level trajectories and propose an efficient solution to solve the high-dimensional assignment problem between non-linear trajectories and events. Their effectiveness is demonstrated in two scenarios: In dense continuous-time motion estimation, our method improves the zero-shot performance of a synthetically trained model on the real-world dataset EVIMO2 by 29%. In optical flow estimation, our method elevates a simple UNet to achieve state-of-the-art performance among self-supervised methods on the DSEC optical flow benchmark. Our code is available at https://github.com/tub-rip/MotionPriorCMax. |
| title | Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation |
| topic | Computer Vision and Pattern Recognition Machine Learning Robotics |
| url | https://arxiv.org/abs/2407.10802 |