Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.01382 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917602915254272 |
|---|---|
| author | Kumar, Rohan Kim, Youngmin Ravi, Sunitha Sun, Haitian Faloutsos, Christos Salakhutdinov, Ruslan Yoon, Minji |
| author_facet | Kumar, Rohan Kim, Youngmin Ravi, Sunitha Sun, Haitian Faloutsos, Christos Salakhutdinov, Ruslan Yoon, Minji |
| contents | Pretrained Large Language Models (LLMs) have gained significant attention for addressing open-domain Question Answering (QA). While they exhibit high accuracy in answering questions related to common knowledge, LLMs encounter difficulties in learning about uncommon long-tail knowledge (tail entities). Since manually constructing QA datasets demands substantial human resources, the types of existing QA datasets are limited, leaving us with a scarcity of datasets to study the performance of LLMs on tail entities. In this paper, we propose an automatic approach to generate specialized QA datasets for tail entities and present the associated research challenges. We conduct extensive experiments by employing pretrained LLMs on our newly generated long-tail QA datasets, comparing their performance with and without external resources including Wikipedia and Wikidata knowledge graphs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_01382 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Automatic Question-Answer Generation for Long-Tail Knowledge Kumar, Rohan Kim, Youngmin Ravi, Sunitha Sun, Haitian Faloutsos, Christos Salakhutdinov, Ruslan Yoon, Minji Computation and Language Pretrained Large Language Models (LLMs) have gained significant attention for addressing open-domain Question Answering (QA). While they exhibit high accuracy in answering questions related to common knowledge, LLMs encounter difficulties in learning about uncommon long-tail knowledge (tail entities). Since manually constructing QA datasets demands substantial human resources, the types of existing QA datasets are limited, leaving us with a scarcity of datasets to study the performance of LLMs on tail entities. In this paper, we propose an automatic approach to generate specialized QA datasets for tail entities and present the associated research challenges. We conduct extensive experiments by employing pretrained LLMs on our newly generated long-tail QA datasets, comparing their performance with and without external resources including Wikipedia and Wikidata knowledge graphs. |
| title | Automatic Question-Answer Generation for Long-Tail Knowledge |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2403.01382 |