Annotation-Free Reinforcement Learning Query Rewriting via Verifiable Search Reward
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908706716778496 |
|---|---|
| author | Cha, Sungguk Kim, DongWook Hahn, Taeseung Kim, Mintae Han, Youngsub Jeon, Byoung-Ki |
| author_facet | Cha, Sungguk Kim, DongWook Hahn, Taeseung Kim, Mintae Han, Youngsub Jeon, Byoung-Ki |
| contents | Optimizing queries for Retrieval-Augmented Generation (RAG) systems poses a significant challenge, particularly across diverse modal indices. We introduce RL-QR, a novel annotation-free reinforcement learning framework for query rewriting that eliminates the need for costly human-annotated data. By leveraging verifiable search rewards derived from index-aligned synthetic queries, RL-QR overcomes human-annotation dependencies, extending its applicability to various modalities and index domains. Experimental results demonstrate the framework's robustness, achieving substantial retrieval performance gains of up to 3.9$\times$ on lexical retrievers and 3.5$\times$ on semantic retrievers on the MTEB VIDORE V2 benchmark for unstructured visual documents, along with consistent 5\% to 10\% improvements on MS MARCO v2.1 and internal industrial datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_23242 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Annotation-Free Reinforcement Learning Query Rewriting via Verifiable Search Reward Cha, Sungguk Kim, DongWook Hahn, Taeseung Kim, Mintae Han, Youngsub Jeon, Byoung-Ki Computer Vision and Pattern Recognition Computation and Language Machine Learning Optimizing queries for Retrieval-Augmented Generation (RAG) systems poses a significant challenge, particularly across diverse modal indices. We introduce RL-QR, a novel annotation-free reinforcement learning framework for query rewriting that eliminates the need for costly human-annotated data. By leveraging verifiable search rewards derived from index-aligned synthetic queries, RL-QR overcomes human-annotation dependencies, extending its applicability to various modalities and index domains. Experimental results demonstrate the framework's robustness, achieving substantial retrieval performance gains of up to 3.9$\times$ on lexical retrievers and 3.5$\times$ on semantic retrievers on the MTEB VIDORE V2 benchmark for unstructured visual documents, along with consistent 5\% to 10\% improvements on MS MARCO v2.1 and internal industrial datasets. |
| title | Annotation-Free Reinforcement Learning Query Rewriting via Verifiable Search Reward |
| topic | Computer Vision and Pattern Recognition Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2507.23242 |