DINOv2: Learning Robust Visual Features without Supervision
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911769328353280 |
|---|---|
| author | Oquab, Maxime Darcet, Timothée Moutakanni, Théo Vo, Huy Szafraniec, Marc Khalidov, Vasil Fernandez, Pierre Haziza, Daniel Massa, Francisco El-Nouby, Alaaeldin Assran, Mahmoud Ballas, Nicolas Galuba, Wojciech Howes, Russell Huang, Po-Yao Li, Shang-Wen Misra, Ishan Rabbat, Michael Sharma, Vasu Synnaeve, Gabriel Xu, Hu Jegou, Hervé Mairal, Julien Labatut, Patrick Joulin, Armand Bojanowski, Piotr |
| author_facet | Oquab, Maxime Darcet, Timothée Moutakanni, Théo Vo, Huy Szafraniec, Marc Khalidov, Vasil Fernandez, Pierre Haziza, Daniel Massa, Francisco El-Nouby, Alaaeldin Assran, Mahmoud Ballas, Nicolas Galuba, Wojciech Howes, Russell Huang, Po-Yao Li, Shang-Wen Misra, Ishan Rabbat, Michael Sharma, Vasu Synnaeve, Gabriel Xu, Hu Jegou, Hervé Mairal, Julien Labatut, Patrick Joulin, Armand Bojanowski, Piotr |
| contents | The recent breakthroughs in natural language processing for model pretraining on large quantities of data have opened the way for similar foundation models in computer vision. These models could greatly simplify the use of images in any system by producing all-purpose visual features, i.e., features that work across image distributions and tasks without finetuning. This work shows that existing pretraining methods, especially self-supervised methods, can produce such features if trained on enough curated data from diverse sources. We revisit existing approaches and combine different techniques to scale our pretraining in terms of data and model size. Most of the technical contributions aim at accelerating and stabilizing the training at scale. In terms of data, we propose an automatic pipeline to build a dedicated, diverse, and curated image dataset instead of uncurated data, as typically done in the self-supervised literature. In terms of models, we train a ViT model (Dosovitskiy et al., 2020) with 1B parameters and distill it into a series of smaller models that surpass the best available all-purpose features, OpenCLIP (Ilharco et al., 2021) on most of the benchmarks at image and pixel levels. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_07193 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | DINOv2: Learning Robust Visual Features without Supervision Oquab, Maxime Darcet, Timothée Moutakanni, Théo Vo, Huy Szafraniec, Marc Khalidov, Vasil Fernandez, Pierre Haziza, Daniel Massa, Francisco El-Nouby, Alaaeldin Assran, Mahmoud Ballas, Nicolas Galuba, Wojciech Howes, Russell Huang, Po-Yao Li, Shang-Wen Misra, Ishan Rabbat, Michael Sharma, Vasu Synnaeve, Gabriel Xu, Hu Jegou, Hervé Mairal, Julien Labatut, Patrick Joulin, Armand Bojanowski, Piotr Computer Vision and Pattern Recognition The recent breakthroughs in natural language processing for model pretraining on large quantities of data have opened the way for similar foundation models in computer vision. These models could greatly simplify the use of images in any system by producing all-purpose visual features, i.e., features that work across image distributions and tasks without finetuning. This work shows that existing pretraining methods, especially self-supervised methods, can produce such features if trained on enough curated data from diverse sources. We revisit existing approaches and combine different techniques to scale our pretraining in terms of data and model size. Most of the technical contributions aim at accelerating and stabilizing the training at scale. In terms of data, we propose an automatic pipeline to build a dedicated, diverse, and curated image dataset instead of uncurated data, as typically done in the self-supervised literature. In terms of models, we train a ViT model (Dosovitskiy et al., 2020) with 1B parameters and distill it into a series of smaller models that surpass the best available all-purpose features, OpenCLIP (Ilharco et al., 2021) on most of the benchmarks at image and pixel levels. |
| title | DINOv2: Learning Robust Visual Features without Supervision |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2304.07193 |