$\text{R}^2\text{R}$: A Route-to-Rerank Post-Training Framework for Multi-Domain Decoder-Only Rerankers
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917102425735168 |
|---|---|
| author | Wang, Xinyu Wu, Hanwei Hu, Qingchen Tai, Zhenghan Tian, Jingrui Ding, Lei Chi, Jijun He, Hailin Kwok, Tung Sum Thomas Cui, Yufei Lyu, Sicheng Li, Muzhi Li, Mingze Yu, Xinyue Zhou, Ling Lu, Peng |
| author_facet | Wang, Xinyu Wu, Hanwei Hu, Qingchen Tai, Zhenghan Tian, Jingrui Ding, Lei Chi, Jijun He, Hailin Kwok, Tung Sum Thomas Cui, Yufei Lyu, Sicheng Li, Muzhi Li, Mingze Yu, Xinyue Zhou, Ling Lu, Peng |
| contents | Decoder-only rerankers are central to Retrieval-Augmented Generation (RAG). However, generalist models miss domain-specific nuances in high-stakes fields like finance and law, and naive fine-tuning causes surface-form overfitting and catastrophic forgetting. To address this challenge, we introduce R2R, a domain-aware framework that combines dynamic expert routing with a two-stage training strategy, Entity Abstraction for Generalization (EAG). EAG introduces a counter-shortcut mechanism by masking the most predictive surface cues, forcing the reranker to learn domain-invariant relevance patterns rather than memorizing dataset-specific entities. To efficiently activate domain experts, R2R employs a lightweight Latent Semantic Router that probes internal representations from the frozen backbone decoder to select the optimal LoRA expert per query. Extensive experiments across different reranker backbones and diverse domains (legal, medical, and financial) demonstrate that R2R consistently surpasses generalist and single-domain fine-tuned baselines. Our results confirm that R2R is a model-agnostic and modular approach to domain specialization with strong cross-domain robustness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_19987 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | $\text{R}^2\text{R}$: A Route-to-Rerank Post-Training Framework for Multi-Domain Decoder-Only Rerankers Wang, Xinyu Wu, Hanwei Hu, Qingchen Tai, Zhenghan Tian, Jingrui Ding, Lei Chi, Jijun He, Hailin Kwok, Tung Sum Thomas Cui, Yufei Lyu, Sicheng Li, Muzhi Li, Mingze Yu, Xinyue Zhou, Ling Lu, Peng Computation and Language Information Retrieval Decoder-only rerankers are central to Retrieval-Augmented Generation (RAG). However, generalist models miss domain-specific nuances in high-stakes fields like finance and law, and naive fine-tuning causes surface-form overfitting and catastrophic forgetting. To address this challenge, we introduce R2R, a domain-aware framework that combines dynamic expert routing with a two-stage training strategy, Entity Abstraction for Generalization (EAG). EAG introduces a counter-shortcut mechanism by masking the most predictive surface cues, forcing the reranker to learn domain-invariant relevance patterns rather than memorizing dataset-specific entities. To efficiently activate domain experts, R2R employs a lightweight Latent Semantic Router that probes internal representations from the frozen backbone decoder to select the optimal LoRA expert per query. Extensive experiments across different reranker backbones and diverse domains (legal, medical, and financial) demonstrate that R2R consistently surpasses generalist and single-domain fine-tuned baselines. Our results confirm that R2R is a model-agnostic and modular approach to domain specialization with strong cross-domain robustness. |
| title | $\text{R}^2\text{R}$: A Route-to-Rerank Post-Training Framework for Multi-Domain Decoder-Only Rerankers |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2511.19987 |