CDG-MAE: Learning Correspondences from Diffusion Generated Views
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916806135906304 |
|---|---|
| author | Belagali, Varun Marza, Pierre Yellapragada, Srikar Li, Zilinghan Nandi, Tarak Nath Madduri, Ravi K Saltz, Joel Christodoulidis, Stergios Vakalopoulou, Maria Samaras, Dimitris |
| author_facet | Belagali, Varun Marza, Pierre Yellapragada, Srikar Li, Zilinghan Nandi, Tarak Nath Madduri, Ravi K Saltz, Joel Christodoulidis, Stergios Vakalopoulou, Maria Samaras, Dimitris |
| contents | Learning dense correspondences, critical for application such as video label propagation, is hindered by tedious and unscalable manual annotation. Self-supervised methods address this by using a cross-view pretext task, often modeled with a masked autoencoder, where a masked target view is reconstructed from an anchor view. However, acquiring effective training data remains a challenge - collecting diverse video datasets is difficult and costly, while simple image crops lack necessary pose variations. This paper introduces CDG-MAE, a novel MAE-based self-supervised method that uses diverse synthetic views generated from static images via an image-conditioned diffusion model. These generated views exhibit substantial changes in pose and perspective, providing a rich training signal that overcomes the limitations of video and crop-based anchors. We present a quantitative method to evaluate local and global consistency of generated images, discussing their use for cross-view self-supervised pretraining. Furthermore, we enhance the standard single-anchor MAE setting to a multi-anchor strategy to effectively modulate the difficulty of pretext task. CDG-MAE significantly outperforms state-of-the-art MAE methods reliant only on images and substantially narrows the performance gap to video-based approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_18164 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CDG-MAE: Learning Correspondences from Diffusion Generated Views Belagali, Varun Marza, Pierre Yellapragada, Srikar Li, Zilinghan Nandi, Tarak Nath Madduri, Ravi K Saltz, Joel Christodoulidis, Stergios Vakalopoulou, Maria Samaras, Dimitris Computer Vision and Pattern Recognition Learning dense correspondences, critical for application such as video label propagation, is hindered by tedious and unscalable manual annotation. Self-supervised methods address this by using a cross-view pretext task, often modeled with a masked autoencoder, where a masked target view is reconstructed from an anchor view. However, acquiring effective training data remains a challenge - collecting diverse video datasets is difficult and costly, while simple image crops lack necessary pose variations. This paper introduces CDG-MAE, a novel MAE-based self-supervised method that uses diverse synthetic views generated from static images via an image-conditioned diffusion model. These generated views exhibit substantial changes in pose and perspective, providing a rich training signal that overcomes the limitations of video and crop-based anchors. We present a quantitative method to evaluate local and global consistency of generated images, discussing their use for cross-view self-supervised pretraining. Furthermore, we enhance the standard single-anchor MAE setting to a multi-anchor strategy to effectively modulate the difficulty of pretext task. CDG-MAE significantly outperforms state-of-the-art MAE methods reliant only on images and substantially narrows the performance gap to video-based approaches. |
| title | CDG-MAE: Learning Correspondences from Diffusion Generated Views |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.18164 |