xVLM2Vec: Adapting LVLM-based embedding models to multilinguality using Self-Knowledge Distillation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910876621078528 |
|---|---|
| author | Musacchio, Elio Siciliani, Lucia Basile, Pierpaolo Semeraro, Giovanni |
| author_facet | Musacchio, Elio Siciliani, Lucia Basile, Pierpaolo Semeraro, Giovanni |
| contents | In the current literature, most embedding models are based on the encoder-only transformer architecture to extract a dense and meaningful representation of the given input, which can be a text, an image, and more. With the recent advances in language modeling thanks to the introduction of Large Language Models, the possibility of extracting embeddings from these large and extensively trained models has been explored. However, current studies focus on textual embeddings in English, which is also the main language on which these models have been trained. Furthermore, there are very few models that consider multimodal and multilingual input. In light of this, we propose an adaptation methodology for Large Vision-Language Models trained on English language data to improve their performance in extracting multilingual and multimodal embeddings. Finally, we design and introduce a benchmark to evaluate the effectiveness of multilingual and multimodal embedding models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_09313 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | xVLM2Vec: Adapting LVLM-based embedding models to multilinguality using Self-Knowledge Distillation Musacchio, Elio Siciliani, Lucia Basile, Pierpaolo Semeraro, Giovanni Computation and Language Information Retrieval In the current literature, most embedding models are based on the encoder-only transformer architecture to extract a dense and meaningful representation of the given input, which can be a text, an image, and more. With the recent advances in language modeling thanks to the introduction of Large Language Models, the possibility of extracting embeddings from these large and extensively trained models has been explored. However, current studies focus on textual embeddings in English, which is also the main language on which these models have been trained. Furthermore, there are very few models that consider multimodal and multilingual input. In light of this, we propose an adaptation methodology for Large Vision-Language Models trained on English language data to improve their performance in extracting multilingual and multimodal embeddings. Finally, we design and introduce a benchmark to evaluate the effectiveness of multilingual and multimodal embedding models. |
| title | xVLM2Vec: Adapting LVLM-based embedding models to multilinguality using Self-Knowledge Distillation |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2503.09313 |