Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.14405 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912288925024256 |
|---|---|
| author | Sariyildiz, Mert Bulent Weinzaepfel, Philippe Lucas, Thomas de Jorge, Pau Larlus, Diane Kalantidis, Yannis |
| author_facet | Sariyildiz, Mert Bulent Weinzaepfel, Philippe Lucas, Thomas de Jorge, Pau Larlus, Diane Kalantidis, Yannis |
| contents | Recent multi-teacher distillation methods have unified the encoders of multiple foundation models into a single encoder, achieving competitive performance on core vision tasks like classification, segmentation, and depth estimation. This led us to ask: Could similar success be achieved when the pool of teachers also includes vision models specialized in diverse tasks across both 2D and 3D perception? In this paper, we define and investigate the problem of heterogeneous teacher distillation, or co-distillation, a challenging multi-teacher distillation scenario where teacher models vary significantly in both (a) their design objectives and (b) the data they were trained on. We explore data-sharing strategies and teacher-specific encoding, and introduce DUNE, a single encoder excelling in 2D vision, 3D understanding, and 3D human perception. Our model achieves performance comparable to that of its larger teachers, sometimes even outperforming them, on their respective tasks. Notably, DUNE surpasses MASt3R in Map-free Visual Relocalization with a much smaller encoder. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_14405 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teachers Sariyildiz, Mert Bulent Weinzaepfel, Philippe Lucas, Thomas de Jorge, Pau Larlus, Diane Kalantidis, Yannis Computer Vision and Pattern Recognition Machine Learning Recent multi-teacher distillation methods have unified the encoders of multiple foundation models into a single encoder, achieving competitive performance on core vision tasks like classification, segmentation, and depth estimation. This led us to ask: Could similar success be achieved when the pool of teachers also includes vision models specialized in diverse tasks across both 2D and 3D perception? In this paper, we define and investigate the problem of heterogeneous teacher distillation, or co-distillation, a challenging multi-teacher distillation scenario where teacher models vary significantly in both (a) their design objectives and (b) the data they were trained on. We explore data-sharing strategies and teacher-specific encoding, and introduce DUNE, a single encoder excelling in 2D vision, 3D understanding, and 3D human perception. Our model achieves performance comparable to that of its larger teachers, sometimes even outperforming them, on their respective tasks. Notably, DUNE surpasses MASt3R in Map-free Visual Relocalization with a much smaller encoder. |
| title | DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teachers |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2503.14405 |