Repurposing Video Diffusion Transformers for Robust Point Tracking
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909974848864256 |
|---|---|
| author | Son, Soowon An, Honggyu Kim, Chaehyun Ko, Hyunah Nam, Jisu Chung, Dahyun Jin, Siyoon Yi, Jung Min, Jaewon Hur, Junhwa Kim, Seungryong |
| author_facet | Son, Soowon An, Honggyu Kim, Chaehyun Ko, Hyunah Nam, Jisu Chung, Dahyun Jin, Siyoon Yi, Jung Min, Jaewon Hur, Junhwa Kim, Seungryong |
| contents | Point tracking aims to localize corresponding points across video frames, serving as a fundamental task for 4D reconstruction, robotics, and video editing. Existing methods commonly rely on shallow convolutional backbones such as ResNet that process frames independently, lacking temporal coherence and producing unreliable matching costs under challenging conditions. Through systematic analysis, we find that video Diffusion Transformers (DiTs), pre-trained on large-scale real-world videos with spatio-temporal attention, inherently exhibit strong point tracking capability and robustly handle dynamic motions and frequent occlusions. We propose DiTracker, which adapts video DiTs through: (1) query-key attention matching, (2) lightweight LoRA tuning, and (3) cost fusion with a ResNet backbone. Despite training with 8 times smaller batch size, DiTracker achieves state-of-the-art performance on challenging ITTO benchmark and matches or outperforms state-of-the-art models on TAP-Vid benchmarks. Our work validates video DiT features as an effective and efficient foundation for point tracking. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_20606 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Repurposing Video Diffusion Transformers for Robust Point Tracking Son, Soowon An, Honggyu Kim, Chaehyun Ko, Hyunah Nam, Jisu Chung, Dahyun Jin, Siyoon Yi, Jung Min, Jaewon Hur, Junhwa Kim, Seungryong Computer Vision and Pattern Recognition Point tracking aims to localize corresponding points across video frames, serving as a fundamental task for 4D reconstruction, robotics, and video editing. Existing methods commonly rely on shallow convolutional backbones such as ResNet that process frames independently, lacking temporal coherence and producing unreliable matching costs under challenging conditions. Through systematic analysis, we find that video Diffusion Transformers (DiTs), pre-trained on large-scale real-world videos with spatio-temporal attention, inherently exhibit strong point tracking capability and robustly handle dynamic motions and frequent occlusions. We propose DiTracker, which adapts video DiTs through: (1) query-key attention matching, (2) lightweight LoRA tuning, and (3) cost fusion with a ResNet backbone. Despite training with 8 times smaller batch size, DiTracker achieves state-of-the-art performance on challenging ITTO benchmark and matches or outperforms state-of-the-art models on TAP-Vid benchmarks. Our work validates video DiT features as an effective and efficient foundation for point tracking. |
| title | Repurposing Video Diffusion Transformers for Robust Point Tracking |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.20606 |