MOVE: A Mixture-of-Vision-Encoders Approach for Domain-Focused Vision-Language Processing
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910838296674304 |
|---|---|
| author | Skripkin, Matvey Goncharova, Elizaveta Tarasov, Dmitrii Kuznetsov, Andrey |
| author_facet | Skripkin, Matvey Goncharova, Elizaveta Tarasov, Dmitrii Kuznetsov, Andrey |
| contents | Multimodal language models (MLMs) integrate visual and textual information by coupling a vision encoder with a large language model through the specific adapter. While existing approaches commonly rely on a single pre-trained vision encoder, there is a great variability of specialized encoders that can boost model's performance in distinct domains. In this work, we propose MOVE (Mixture of Vision Encoders) a simple yet effective approach to leverage multiple pre-trained encoders for specialized multimodal tasks. MOVE automatically routes inputs to the most appropriate encoder among candidates such as Unichat, InternViT, and Texify, thereby enhancing performance across a diverse set of benchmarks, including ChartQA, MMBench, and MMMU. Experimental results demonstrate that MOVE achieves competitive accuracy without incurring the complexities of image slicing for high-resolution images. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_15381 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MOVE: A Mixture-of-Vision-Encoders Approach for Domain-Focused Vision-Language Processing Skripkin, Matvey Goncharova, Elizaveta Tarasov, Dmitrii Kuznetsov, Andrey Computer Vision and Pattern Recognition 6804, 68T50 (Primary) I.2.7; I.2.10; I.4.9 Multimodal language models (MLMs) integrate visual and textual information by coupling a vision encoder with a large language model through the specific adapter. While existing approaches commonly rely on a single pre-trained vision encoder, there is a great variability of specialized encoders that can boost model's performance in distinct domains. In this work, we propose MOVE (Mixture of Vision Encoders) a simple yet effective approach to leverage multiple pre-trained encoders for specialized multimodal tasks. MOVE automatically routes inputs to the most appropriate encoder among candidates such as Unichat, InternViT, and Texify, thereby enhancing performance across a diverse set of benchmarks, including ChartQA, MMBench, and MMMU. Experimental results demonstrate that MOVE achieves competitive accuracy without incurring the complexities of image slicing for high-resolution images. |
| title | MOVE: A Mixture-of-Vision-Encoders Approach for Domain-Focused Vision-Language Processing |
| topic | Computer Vision and Pattern Recognition 6804, 68T50 (Primary) I.2.7; I.2.10; I.4.9 |
| url | https://arxiv.org/abs/2502.15381 |