Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2509.00276 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914014414503936 |
|---|---|
| author | Liu, Yuxiang Wang, Tian Kundu, Gourab Cao, Tianyu Cheng, Guang Ge, Zhen Chen, Jianshu Cui, Qingjun Chilimbi, Trishul |
| author_facet | Liu, Yuxiang Wang, Tian Kundu, Gourab Cao, Tianyu Cheng, Guang Ge, Zhen Chen, Jianshu Cui, Qingjun Chilimbi, Trishul |
| contents | Transformer-based models such as BERT and E5 have significantly advanced text embedding by capturing rich contextual representations. However, many complex real-world queries require sophisticated reasoning to retrieve relevant documents beyond surface-level lexical matching, where encoder-only retrievers often fall short. Decoder-only large language models (LLMs), known for their strong reasoning capabilities, offer a promising alternative. Despite this potential, existing LLM-based embedding methods primarily focus on contextual representation and do not fully exploit the reasoning strength of LLMs. To bridge this gap, we propose Reasoning-Infused Text Embedding (RITE), a simple but effective approach that integrates logical reasoning into the text embedding process using generative LLMs. RITE builds upon existing language model embedding techniques by generating intermediate reasoning texts in the token space before computing embeddings, thereby enriching representations with inferential depth. Experimental results on BRIGHT, a reasoning-intensive retrieval benchmark, demonstrate that RITE significantly enhances zero-shot retrieval performance across diverse domains, underscoring the effectiveness of incorporating reasoning into the embedding process. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_00276 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploring Reasoning-Infused Text Embedding with Large Language Models for Zero-Shot Dense Retrieval Liu, Yuxiang Wang, Tian Kundu, Gourab Cao, Tianyu Cheng, Guang Ge, Zhen Chen, Jianshu Cui, Qingjun Chilimbi, Trishul Computation and Language Transformer-based models such as BERT and E5 have significantly advanced text embedding by capturing rich contextual representations. However, many complex real-world queries require sophisticated reasoning to retrieve relevant documents beyond surface-level lexical matching, where encoder-only retrievers often fall short. Decoder-only large language models (LLMs), known for their strong reasoning capabilities, offer a promising alternative. Despite this potential, existing LLM-based embedding methods primarily focus on contextual representation and do not fully exploit the reasoning strength of LLMs. To bridge this gap, we propose Reasoning-Infused Text Embedding (RITE), a simple but effective approach that integrates logical reasoning into the text embedding process using generative LLMs. RITE builds upon existing language model embedding techniques by generating intermediate reasoning texts in the token space before computing embeddings, thereby enriching representations with inferential depth. Experimental results on BRIGHT, a reasoning-intensive retrieval benchmark, demonstrate that RITE significantly enhances zero-shot retrieval performance across diverse domains, underscoring the effectiveness of incorporating reasoning into the embedding process. |
| title | Exploring Reasoning-Infused Text Embedding with Large Language Models for Zero-Shot Dense Retrieval |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.00276 |