Alleviating LLM-based Generative Retrieval Hallucination in Alipay Search
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916734519214080 |
|---|---|
| author | Shen, Yedan Wu, Kaixin Ding, Yuechen Wen, Jingyuan Liu, Hong Zhong, Mingjie Lin, Zhouhan Xu, Jia Mo, Linjian |
| author_facet | Shen, Yedan Wu, Kaixin Ding, Yuechen Wen, Jingyuan Liu, Hong Zhong, Mingjie Lin, Zhouhan Xu, Jia Mo, Linjian |
| contents | Generative retrieval (GR) has revolutionized document retrieval with the advent of large language models (LLMs), and LLM-based GR is gradually being adopted by the industry. Despite its remarkable advantages and potential, LLM-based GR suffers from hallucination and generates documents that are irrelevant to the query in some instances, severely challenging its credibility in practical applications. We thereby propose an optimized GR framework designed to alleviate retrieval hallucination, which integrates knowledge distillation reasoning in model training and incorporate decision agent to further improve retrieval precision. Specifically, we employ LLMs to assess and reason GR retrieved query-document (q-d) pairs, and then distill the reasoning data as transferred knowledge to the GR model. Moreover, we utilize a decision agent as post-processing to extend the GR retrieved documents through retrieval model and select the most relevant ones from multi perspectives as the final generative retrieval result. Extensive offline experiments on real-world datasets and online A/B tests on Fund Search and Insurance Search in Alipay demonstrate our framework's superiority and effectiveness in improving search quality and conversion gains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_21098 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Alleviating LLM-based Generative Retrieval Hallucination in Alipay Search Shen, Yedan Wu, Kaixin Ding, Yuechen Wen, Jingyuan Liu, Hong Zhong, Mingjie Lin, Zhouhan Xu, Jia Mo, Linjian Information Retrieval Artificial Intelligence Generative retrieval (GR) has revolutionized document retrieval with the advent of large language models (LLMs), and LLM-based GR is gradually being adopted by the industry. Despite its remarkable advantages and potential, LLM-based GR suffers from hallucination and generates documents that are irrelevant to the query in some instances, severely challenging its credibility in practical applications. We thereby propose an optimized GR framework designed to alleviate retrieval hallucination, which integrates knowledge distillation reasoning in model training and incorporate decision agent to further improve retrieval precision. Specifically, we employ LLMs to assess and reason GR retrieved query-document (q-d) pairs, and then distill the reasoning data as transferred knowledge to the GR model. Moreover, we utilize a decision agent as post-processing to extend the GR retrieved documents through retrieval model and select the most relevant ones from multi perspectives as the final generative retrieval result. Extensive offline experiments on real-world datasets and online A/B tests on Fund Search and Insurance Search in Alipay demonstrate our framework's superiority and effectiveness in improving search quality and conversion gains. |
| title | Alleviating LLM-based Generative Retrieval Hallucination in Alipay Search |
| topic | Information Retrieval Artificial Intelligence |
| url | https://arxiv.org/abs/2503.21098 |