Cite Before You Speak: Enhancing Context-Response Grounding in E-commerce Conversational LLM-Agents
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908361295921152 |
|---|---|
| author | Zeng, Jingying Liu, Hui Dai, Zhenwei Tang, Xianfeng Luo, Chen Varshney, Samarth Li, Zhen He, Qi |
| author_facet | Zeng, Jingying Liu, Hui Dai, Zhenwei Tang, Xianfeng Luo, Chen Varshney, Samarth Li, Zhen He, Qi |
| contents | With the advancement of conversational large language models (LLMs), several LLM-based Conversational Shopping Agents (CSA) have been developed to help customers smooth their online shopping. The primary objective in building an engaging and trustworthy CSA is to ensure the agent's responses about product factoids are accurate and factually grounded. However, two challenges remain. First, LLMs produce hallucinated or unsupported claims. Such inaccuracies risk spreading misinformation and diminishing customer trust. Second, without providing knowledge source attribution in CSA response, customers struggle to verify LLM-generated information. To address both challenges, we present an easily productionized solution that enables a ''citation experience'' to our customers. We build auto-evaluation metrics to holistically evaluate LLM's grounding and attribution capabilities, suggesting that citation generation paradigm substantially improves grounding performance by 13.83%. To deploy this capability at scale, we introduce Multi-UX-Inference system, which appends source citations to LLM outputs while preserving existing user experience features and supporting scalable inference. Large-scale online A/B tests show that grounded CSA responses improves customer engagement by 3% - 10%, depending on UX variations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_04830 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Cite Before You Speak: Enhancing Context-Response Grounding in E-commerce Conversational LLM-Agents Zeng, Jingying Liu, Hui Dai, Zhenwei Tang, Xianfeng Luo, Chen Varshney, Samarth Li, Zhen He, Qi Computation and Language Artificial Intelligence With the advancement of conversational large language models (LLMs), several LLM-based Conversational Shopping Agents (CSA) have been developed to help customers smooth their online shopping. The primary objective in building an engaging and trustworthy CSA is to ensure the agent's responses about product factoids are accurate and factually grounded. However, two challenges remain. First, LLMs produce hallucinated or unsupported claims. Such inaccuracies risk spreading misinformation and diminishing customer trust. Second, without providing knowledge source attribution in CSA response, customers struggle to verify LLM-generated information. To address both challenges, we present an easily productionized solution that enables a ''citation experience'' to our customers. We build auto-evaluation metrics to holistically evaluate LLM's grounding and attribution capabilities, suggesting that citation generation paradigm substantially improves grounding performance by 13.83%. To deploy this capability at scale, we introduce Multi-UX-Inference system, which appends source citations to LLM outputs while preserving existing user experience features and supporting scalable inference. Large-scale online A/B tests show that grounded CSA responses improves customer engagement by 3% - 10%, depending on UX variations. |
| title | Cite Before You Speak: Enhancing Context-Response Grounding in E-commerce Conversational LLM-Agents |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.04830 |