Retrieval Augmented Generation for Domain-specific Question Answering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917678280605696 |
|---|---|
| author | Sharma, Sanat Yoon, David Seunghyun Dernoncourt, Franck Sultania, Dewang Bagga, Karishma Zhang, Mengjiao Bui, Trung Kotte, Varun |
| author_facet | Sharma, Sanat Yoon, David Seunghyun Dernoncourt, Franck Sultania, Dewang Bagga, Karishma Zhang, Mengjiao Bui, Trung Kotte, Varun |
| contents | Question answering (QA) has become an important application in the advanced development of large language models. General pre-trained large language models for question-answering are not trained to properly understand the knowledge or terminology for a specific domain, such as finance, healthcare, education, and customer service for a product. To better cater to domain-specific understanding, we build an in-house question-answering system for Adobe products. We propose a novel framework to compile a large question-answer database and develop the approach for retrieval-aware finetuning of a Large Language model. We showcase that fine-tuning the retriever leads to major improvements in the final generation. Our overall approach reduces hallucinations during generation while keeping in context the latest retrieval information for contextual grounding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_14760 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Retrieval Augmented Generation for Domain-specific Question Answering Sharma, Sanat Yoon, David Seunghyun Dernoncourt, Franck Sultania, Dewang Bagga, Karishma Zhang, Mengjiao Bui, Trung Kotte, Varun Computation and Language Artificial Intelligence Information Retrieval Machine Learning Question answering (QA) has become an important application in the advanced development of large language models. General pre-trained large language models for question-answering are not trained to properly understand the knowledge or terminology for a specific domain, such as finance, healthcare, education, and customer service for a product. To better cater to domain-specific understanding, we build an in-house question-answering system for Adobe products. We propose a novel framework to compile a large question-answer database and develop the approach for retrieval-aware finetuning of a Large Language model. We showcase that fine-tuning the retriever leads to major improvements in the final generation. Our overall approach reduces hallucinations during generation while keeping in context the latest retrieval information for contextual grounding. |
| title | Retrieval Augmented Generation for Domain-specific Question Answering |
| topic | Computation and Language Artificial Intelligence Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2404.14760 |