Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.05606 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911910702612480 |
|---|---|
| author | Ko, Dayoon Kim, Jinyoung Choi, Hahyeon Kim, Gunhee |
| author_facet | Ko, Dayoon Kim, Jinyoung Choi, Hahyeon Kim, Gunhee |
| contents | In the real world, knowledge is constantly evolving, which can render existing knowledge-based datasets outdated. This unreliability highlights the critical need for continuous updates to ensure both accuracy and relevance in knowledge-intensive tasks. To address this, we propose GrowOVER-QA and GrowOVER-Dialogue, dynamic open-domain QA and dialogue benchmarks that undergo a continuous cycle of updates, keeping pace with the rapid evolution of knowledge. Our research indicates that retrieval-augmented language models (RaLMs) struggle with knowledge that has not been trained on or recently updated. Consequently, we introduce a novel retrieval-interactive language model framework, where the language model evaluates and reflects on its answers for further re-retrieval. Our exhaustive experiments demonstrate that our training-free framework significantly improves upon existing methods, performing comparably to or even surpassing continuously trained language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_05606 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GrowOVER: How Can LLMs Adapt to Growing Real-World Knowledge? Ko, Dayoon Kim, Jinyoung Choi, Hahyeon Kim, Gunhee Computation and Language In the real world, knowledge is constantly evolving, which can render existing knowledge-based datasets outdated. This unreliability highlights the critical need for continuous updates to ensure both accuracy and relevance in knowledge-intensive tasks. To address this, we propose GrowOVER-QA and GrowOVER-Dialogue, dynamic open-domain QA and dialogue benchmarks that undergo a continuous cycle of updates, keeping pace with the rapid evolution of knowledge. Our research indicates that retrieval-augmented language models (RaLMs) struggle with knowledge that has not been trained on or recently updated. Consequently, we introduce a novel retrieval-interactive language model framework, where the language model evaluates and reflects on its answers for further re-retrieval. Our exhaustive experiments demonstrate that our training-free framework significantly improves upon existing methods, performing comparably to or even surpassing continuously trained language models. |
| title | GrowOVER: How Can LLMs Adapt to Growing Real-World Knowledge? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.05606 |