Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2410.24211 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909516017172480 |
|---|---|
| author | Ngo, Tuan Duc Zhuang, Peiye Gan, Chuang Kalogerakis, Evangelos Tulyakov, Sergey Lee, Hsin-Ying Wang, Chaoyang |
| author_facet | Ngo, Tuan Duc Zhuang, Peiye Gan, Chuang Kalogerakis, Evangelos Tulyakov, Sergey Lee, Hsin-Ying Wang, Chaoyang |
| contents | Tracking dense 3D motion from monocular videos remains challenging, particularly when aiming for pixel-level precision over long sequences. We introduce DELTA, a novel method that efficiently tracks every pixel in 3D space, enabling accurate motion estimation across entire videos. Our approach leverages a joint global-local attention mechanism for reduced-resolution tracking, followed by a transformer-based upsampler to achieve high-resolution predictions. Unlike existing methods, which are limited by computational inefficiency or sparse tracking, DELTA delivers dense 3D tracking at scale, running over 8x faster than previous methods while achieving state-of-the-art accuracy. Furthermore, we explore the impact of depth representation on tracking performance and identify log-depth as the optimal choice. Extensive experiments demonstrate the superiority of DELTA on multiple benchmarks, achieving new state-of-the-art results in both 2D and 3D dense tracking tasks. Our method provides a robust solution for applications requiring fine-grained, long-term motion tracking in 3D space. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_24211 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DELTA: Dense Efficient Long-range 3D Tracking for any video Ngo, Tuan Duc Zhuang, Peiye Gan, Chuang Kalogerakis, Evangelos Tulyakov, Sergey Lee, Hsin-Ying Wang, Chaoyang Computer Vision and Pattern Recognition Tracking dense 3D motion from monocular videos remains challenging, particularly when aiming for pixel-level precision over long sequences. We introduce DELTA, a novel method that efficiently tracks every pixel in 3D space, enabling accurate motion estimation across entire videos. Our approach leverages a joint global-local attention mechanism for reduced-resolution tracking, followed by a transformer-based upsampler to achieve high-resolution predictions. Unlike existing methods, which are limited by computational inefficiency or sparse tracking, DELTA delivers dense 3D tracking at scale, running over 8x faster than previous methods while achieving state-of-the-art accuracy. Furthermore, we explore the impact of depth representation on tracking performance and identify log-depth as the optimal choice. Extensive experiments demonstrate the superiority of DELTA on multiple benchmarks, achieving new state-of-the-art results in both 2D and 3D dense tracking tasks. Our method provides a robust solution for applications requiring fine-grained, long-term motion tracking in 3D space. |
| title | DELTA: Dense Efficient Long-range 3D Tracking for any video |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.24211 |