SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913700814782464 |
|---|---|
| author | Tschannen, Michael Gritsenko, Alexey Wang, Xiao Naeem, Muhammad Ferjad Alabdulmohsin, Ibrahim Parthasarathy, Nikhil Evans, Talfan Beyer, Lucas Xia, Ye Mustafa, Basil Hénaff, Olivier Harmsen, Jeremiah Steiner, Andreas Zhai, Xiaohua |
| author_facet | Tschannen, Michael Gritsenko, Alexey Wang, Xiao Naeem, Muhammad Ferjad Alabdulmohsin, Ibrahim Parthasarathy, Nikhil Evans, Talfan Beyer, Lucas Xia, Ye Mustafa, Basil Hénaff, Olivier Harmsen, Jeremiah Steiner, Andreas Zhai, Xiaohua |
| contents | We introduce SigLIP 2, a family of new multilingual vision-language encoders that build on the success of the original SigLIP. In this second iteration, we extend the original image-text training objective with several prior, independently developed techniques into a unified recipe -- this includes captioning-based pretraining, self-supervised losses (self-distillation, masked prediction) and online data curation. With these changes, SigLIP 2 models outperform their SigLIP counterparts at all model scales in core capabilities, including zero-shot classification, image-text retrieval, and transfer performance when extracting visual representations for Vision-Language Models (VLMs). Furthermore, the new training recipe leads to significant improvements on localization and dense prediction tasks. We also train variants which support multiple resolutions and preserve the input's native aspect ratio. Finally, we train on a more diverse data-mixture that includes de-biasing techniques, leading to much better multilingual understanding and improved fairness. To allow users to trade off inference cost with performance, we release model checkpoints at four sizes: ViT-B (86M), L (303M), So400m (400M), and g (1B). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_14786 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features Tschannen, Michael Gritsenko, Alexey Wang, Xiao Naeem, Muhammad Ferjad Alabdulmohsin, Ibrahim Parthasarathy, Nikhil Evans, Talfan Beyer, Lucas Xia, Ye Mustafa, Basil Hénaff, Olivier Harmsen, Jeremiah Steiner, Andreas Zhai, Xiaohua Computer Vision and Pattern Recognition Artificial Intelligence We introduce SigLIP 2, a family of new multilingual vision-language encoders that build on the success of the original SigLIP. In this second iteration, we extend the original image-text training objective with several prior, independently developed techniques into a unified recipe -- this includes captioning-based pretraining, self-supervised losses (self-distillation, masked prediction) and online data curation. With these changes, SigLIP 2 models outperform their SigLIP counterparts at all model scales in core capabilities, including zero-shot classification, image-text retrieval, and transfer performance when extracting visual representations for Vision-Language Models (VLMs). Furthermore, the new training recipe leads to significant improvements on localization and dense prediction tasks. We also train variants which support multiple resolutions and preserve the input's native aspect ratio. Finally, we train on a more diverse data-mixture that includes de-biasing techniques, leading to much better multilingual understanding and improved fairness. To allow users to trade off inference cost with performance, we release model checkpoints at four sizes: ViT-B (86M), L (303M), So400m (400M), and g (1B). |
| title | SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2502.14786 |