Text Embeddings by Weakly-Supervised Contrastive Pre-training
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2022
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911781645975552 |
|---|---|
| author | Wang, Liang Yang, Nan Huang, Xiaolong Jiao, Binxing Yang, Linjun Jiang, Daxin Majumder, Rangan Wei, Furu |
| author_facet | Wang, Liang Yang, Nan Huang, Xiaolong Jiao, Binxing Yang, Linjun Jiang, Daxin Majumder, Rangan Wei, Furu |
| contents | This paper presents E5, a family of state-of-the-art text embeddings that transfer well to a wide range of tasks. The model is trained in a contrastive manner with weak supervision signals from our curated large-scale text pair dataset (called CCPairs). E5 can be readily used as a general-purpose embedding model for any tasks requiring a single-vector representation of texts such as retrieval, clustering, and classification, achieving strong performance in both zero-shot and fine-tuned settings. We conduct extensive evaluations on 56 datasets from the BEIR and MTEB benchmarks. For zero-shot settings, E5 is the first model that outperforms the strong BM25 baseline on the BEIR retrieval benchmark without using any labeled data. When fine-tuned, E5 obtains the best results on the MTEB benchmark, beating existing embedding models with 40x more parameters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2212_03533 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Text Embeddings by Weakly-Supervised Contrastive Pre-training Wang, Liang Yang, Nan Huang, Xiaolong Jiao, Binxing Yang, Linjun Jiang, Daxin Majumder, Rangan Wei, Furu Computation and Language Information Retrieval This paper presents E5, a family of state-of-the-art text embeddings that transfer well to a wide range of tasks. The model is trained in a contrastive manner with weak supervision signals from our curated large-scale text pair dataset (called CCPairs). E5 can be readily used as a general-purpose embedding model for any tasks requiring a single-vector representation of texts such as retrieval, clustering, and classification, achieving strong performance in both zero-shot and fine-tuned settings. We conduct extensive evaluations on 56 datasets from the BEIR and MTEB benchmarks. For zero-shot settings, E5 is the first model that outperforms the strong BM25 baseline on the BEIR retrieval benchmark without using any labeled data. When fine-tuned, E5 obtains the best results on the MTEB benchmark, beating existing embedding models with 40x more parameters. |
| title | Text Embeddings by Weakly-Supervised Contrastive Pre-training |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2212.03533 |