KaPQA: Knowledge-Augmented Product Question-Answering
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913441790296064 |
|---|---|
| author | Eppalapally, Swetha Dangi, Daksh Bhat, Chaithra Gupta, Ankita Zhang, Ruiyi Agarwal, Shubham Bagga, Karishma Yoon, Seunghyun Lipka, Nedim Rossi, Ryan A. Dernoncourt, Franck |
| author_facet | Eppalapally, Swetha Dangi, Daksh Bhat, Chaithra Gupta, Ankita Zhang, Ruiyi Agarwal, Shubham Bagga, Karishma Yoon, Seunghyun Lipka, Nedim Rossi, Ryan A. Dernoncourt, Franck |
| contents | Question-answering for domain-specific applications has recently attracted much interest due to the latest advancements in large language models (LLMs). However, accurately assessing the performance of these applications remains a challenge, mainly due to the lack of suitable benchmarks that effectively simulate real-world scenarios. To address this challenge, we introduce two product question-answering (QA) datasets focused on Adobe Acrobat and Photoshop products to help evaluate the performance of existing models on domain-specific product QA tasks. Additionally, we propose a novel knowledge-driven RAG-QA framework to enhance the performance of the models in the product QA task. Our experiments demonstrated that inducing domain knowledge through query reformulation allowed for increased retrieval and generative performance when compared to standard RAG-QA methods. This improvement, however, is slight, and thus illustrates the challenge posed by the datasets introduced. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_16073 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | KaPQA: Knowledge-Augmented Product Question-Answering Eppalapally, Swetha Dangi, Daksh Bhat, Chaithra Gupta, Ankita Zhang, Ruiyi Agarwal, Shubham Bagga, Karishma Yoon, Seunghyun Lipka, Nedim Rossi, Ryan A. Dernoncourt, Franck Computation and Language Question-answering for domain-specific applications has recently attracted much interest due to the latest advancements in large language models (LLMs). However, accurately assessing the performance of these applications remains a challenge, mainly due to the lack of suitable benchmarks that effectively simulate real-world scenarios. To address this challenge, we introduce two product question-answering (QA) datasets focused on Adobe Acrobat and Photoshop products to help evaluate the performance of existing models on domain-specific product QA tasks. Additionally, we propose a novel knowledge-driven RAG-QA framework to enhance the performance of the models in the product QA task. Our experiments demonstrated that inducing domain knowledge through query reformulation allowed for increased retrieval and generative performance when compared to standard RAG-QA methods. This improvement, however, is slight, and thus illustrates the challenge posed by the datasets introduced. |
| title | KaPQA: Knowledge-Augmented Product Question-Answering |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2407.16073 |