R3A: Reinforced Reasoning for Relevance Assessment for RAG in User-Generated Content Platforms
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910144527335424 |
|---|---|
| author | Yuan, Xiaowei Jin, Lei Zhang, Haoxin Huang, Ziyang Gao, Yan Wu, Yi Hu, Yao Zhao, Jun Liu, Kang |
| author_facet | Yuan, Xiaowei Jin, Lei Zhang, Haoxin Huang, Ziyang Gao, Yan Wu, Yi Hu, Yao Zhao, Jun Liu, Kang |
| contents | Retrieval-augmented generation (RAG) plays a critical role in user-generated content (UGC) platforms, but its effectiveness critically depends on accurate query-document relevance assessment. Despite recent advances in applying large language models (LLMs) to relevance modeling, UGC platforms present unique challenges: 1) ambiguous user intent due to sparse user feedback in RAG scenarios, and 2) asymmetric relevance, where relevance is driven by localized answer-bearing content rather than global query-document similarity. To address these issues, we propose the Reinforced Reasoning model for Relevance Assessment (R3A), which decomposes relevance assessment into intent inference and evidence grounding. R3A leverages auxiliary high-clicked documents to infer latent query intent, and extracts verbatim evidence fragments to ground relevance decisions, reducing noise sensitivity and improving asymmetric relevance modeling. Experimental results demonstrate that R3A substantially outperforms strong baselines on offline benchmarks, while the distilled R3A-1.5B model achieves significant gains in large-scale online A/B testing, effectively balancing performance and practical deployability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_02506 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | R3A: Reinforced Reasoning for Relevance Assessment for RAG in User-Generated Content Platforms Yuan, Xiaowei Jin, Lei Zhang, Haoxin Huang, Ziyang Gao, Yan Wu, Yi Hu, Yao Zhao, Jun Liu, Kang Information Retrieval Artificial Intelligence Retrieval-augmented generation (RAG) plays a critical role in user-generated content (UGC) platforms, but its effectiveness critically depends on accurate query-document relevance assessment. Despite recent advances in applying large language models (LLMs) to relevance modeling, UGC platforms present unique challenges: 1) ambiguous user intent due to sparse user feedback in RAG scenarios, and 2) asymmetric relevance, where relevance is driven by localized answer-bearing content rather than global query-document similarity. To address these issues, we propose the Reinforced Reasoning model for Relevance Assessment (R3A), which decomposes relevance assessment into intent inference and evidence grounding. R3A leverages auxiliary high-clicked documents to infer latent query intent, and extracts verbatim evidence fragments to ground relevance decisions, reducing noise sensitivity and improving asymmetric relevance modeling. Experimental results demonstrate that R3A substantially outperforms strong baselines on offline benchmarks, while the distilled R3A-1.5B model achieves significant gains in large-scale online A/B testing, effectively balancing performance and practical deployability. |
| title | R3A: Reinforced Reasoning for Relevance Assessment for RAG in User-Generated Content Platforms |
| topic | Information Retrieval Artificial Intelligence |
| url | https://arxiv.org/abs/2508.02506 |