jina-embeddings-v3: Multilingual Embeddings With Task LoRA
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914952542945280 |
|---|---|
| author | Sturua, Saba Mohr, Isabelle Akram, Mohammad Kalim Günther, Michael Wang, Bo Krimmel, Markus Wang, Feng Mastrapas, Georgios Koukounas, Andreas Wang, Nan Xiao, Han |
| author_facet | Sturua, Saba Mohr, Isabelle Akram, Mohammad Kalim Günther, Michael Wang, Bo Krimmel, Markus Wang, Feng Mastrapas, Georgios Koukounas, Andreas Wang, Nan Xiao, Han |
| contents | We introduce jina-embeddings-v3, a novel text embedding model with 570 million parameters, achieves state-of-the-art performance on multilingual data and long-context retrieval tasks, supporting context lengths of up to 8192 tokens. The model includes a set of task-specific Low-Rank Adaptation (LoRA) adapters to generate high-quality embeddings for query-document retrieval, clustering, classification, and text matching. Evaluation on the MTEB benchmark shows that jina-embeddings-v3 outperforms the latest proprietary embeddings from OpenAI and Cohere on English tasks, while achieving superior performance compared to multilingual-e5-large-instruct across all multilingual tasks. With a default output dimension of 1024, users can flexibly reduce the embedding dimensions to as low as 32 without compromising performance, enabled by Matryoshka Representation Learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_10173 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | jina-embeddings-v3: Multilingual Embeddings With Task LoRA Sturua, Saba Mohr, Isabelle Akram, Mohammad Kalim Günther, Michael Wang, Bo Krimmel, Markus Wang, Feng Mastrapas, Georgios Koukounas, Andreas Wang, Nan Xiao, Han Computation and Language Artificial Intelligence Information Retrieval 68T50 I.2.7 We introduce jina-embeddings-v3, a novel text embedding model with 570 million parameters, achieves state-of-the-art performance on multilingual data and long-context retrieval tasks, supporting context lengths of up to 8192 tokens. The model includes a set of task-specific Low-Rank Adaptation (LoRA) adapters to generate high-quality embeddings for query-document retrieval, clustering, classification, and text matching. Evaluation on the MTEB benchmark shows that jina-embeddings-v3 outperforms the latest proprietary embeddings from OpenAI and Cohere on English tasks, while achieving superior performance compared to multilingual-e5-large-instruct across all multilingual tasks. With a default output dimension of 1024, users can flexibly reduce the embedding dimensions to as low as 32 without compromising performance, enabled by Matryoshka Representation Learning. |
| title | jina-embeddings-v3: Multilingual Embeddings With Task LoRA |
| topic | Computation and Language Artificial Intelligence Information Retrieval 68T50 I.2.7 |
| url | https://arxiv.org/abs/2409.10173 |