Golden-Retriever: High-Fidelity Agentic Retrieval Augmented Generation for Industrial Knowledge Base
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910551850876928 |
|---|---|
| author | An, Zhiyu Ding, Xianzhong Fu, Yen-Chun Chu, Cheng-Chung Li, Yan Du, Wan |
| author_facet | An, Zhiyu Ding, Xianzhong Fu, Yen-Chun Chu, Cheng-Chung Li, Yan Du, Wan |
| contents | This paper introduces Golden-Retriever, designed to efficiently navigate vast industrial knowledge bases, overcoming challenges in traditional LLM fine-tuning and RAG frameworks with domain-specific jargon and context interpretation. Golden-Retriever incorporates a reflection-based question augmentation step before document retrieval, which involves identifying jargon, clarifying its meaning based on context, and augmenting the question accordingly. Specifically, our method extracts and lists all jargon and abbreviations in the input question, determines the context against a pre-defined list, and queries a jargon dictionary for extended definitions and descriptions. This comprehensive augmentation ensures the RAG framework retrieves the most relevant documents by providing clear context and resolving ambiguities, significantly improving retrieval accuracy. Evaluations using three open-source LLMs on a domain-specific question-answer dataset demonstrate Golden-Retriever's superior performance, providing a robust solution for efficiently integrating and querying industrial knowledge bases. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_00798 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Golden-Retriever: High-Fidelity Agentic Retrieval Augmented Generation for Industrial Knowledge Base An, Zhiyu Ding, Xianzhong Fu, Yen-Chun Chu, Cheng-Chung Li, Yan Du, Wan Information Retrieval Artificial Intelligence Computation and Language Digital Libraries This paper introduces Golden-Retriever, designed to efficiently navigate vast industrial knowledge bases, overcoming challenges in traditional LLM fine-tuning and RAG frameworks with domain-specific jargon and context interpretation. Golden-Retriever incorporates a reflection-based question augmentation step before document retrieval, which involves identifying jargon, clarifying its meaning based on context, and augmenting the question accordingly. Specifically, our method extracts and lists all jargon and abbreviations in the input question, determines the context against a pre-defined list, and queries a jargon dictionary for extended definitions and descriptions. This comprehensive augmentation ensures the RAG framework retrieves the most relevant documents by providing clear context and resolving ambiguities, significantly improving retrieval accuracy. Evaluations using three open-source LLMs on a domain-specific question-answer dataset demonstrate Golden-Retriever's superior performance, providing a robust solution for efficiently integrating and querying industrial knowledge bases. |
| title | Golden-Retriever: High-Fidelity Agentic Retrieval Augmented Generation for Industrial Knowledge Base |
| topic | Information Retrieval Artificial Intelligence Computation and Language Digital Libraries |
| url | https://arxiv.org/abs/2408.00798 |