Evaluating the External and Parametric Knowledge Fusion of Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913368446599168 |
|---|---|
| author | Zhang, Hao Zhang, Yuyang Li, Xiaoguang Shi, Wenxuan Xu, Haonan Liu, Huanshuo Wang, Yasheng Shang, Lifeng Liu, Qun Liu, Yong Tang, Ruiming |
| author_facet | Zhang, Hao Zhang, Yuyang Li, Xiaoguang Shi, Wenxuan Xu, Haonan Liu, Huanshuo Wang, Yasheng Shang, Lifeng Liu, Qun Liu, Yong Tang, Ruiming |
| contents | Integrating external knowledge into large language models (LLMs) presents a promising solution to overcome the limitations imposed by their antiquated and static parametric memory. Prior studies, however, have tended to over-reliance on external knowledge, underestimating the valuable contributions of an LLMs' intrinsic parametric knowledge. The efficacy of LLMs in blending external and parametric knowledge remains largely unexplored, especially in cases where external knowledge is incomplete and necessitates supplementation by their parametric knowledge. We propose to deconstruct knowledge fusion into four distinct scenarios, offering the first thorough investigation of LLM behavior across each. We develop a systematic pipeline for data construction and knowledge infusion to simulate these fusion scenarios, facilitating a series of controlled experiments. Our investigation reveals that enhancing parametric knowledge within LLMs can significantly bolster their capability for knowledge integration. Nonetheless, we identify persistent challenges in memorizing and eliciting parametric knowledge, and determining parametric knowledge boundaries. Our findings aim to steer future explorations on harmonizing external and parametric knowledge within LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_19010 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Evaluating the External and Parametric Knowledge Fusion of Large Language Models Zhang, Hao Zhang, Yuyang Li, Xiaoguang Shi, Wenxuan Xu, Haonan Liu, Huanshuo Wang, Yasheng Shang, Lifeng Liu, Qun Liu, Yong Tang, Ruiming Computation and Language Artificial Intelligence Information Retrieval Integrating external knowledge into large language models (LLMs) presents a promising solution to overcome the limitations imposed by their antiquated and static parametric memory. Prior studies, however, have tended to over-reliance on external knowledge, underestimating the valuable contributions of an LLMs' intrinsic parametric knowledge. The efficacy of LLMs in blending external and parametric knowledge remains largely unexplored, especially in cases where external knowledge is incomplete and necessitates supplementation by their parametric knowledge. We propose to deconstruct knowledge fusion into four distinct scenarios, offering the first thorough investigation of LLM behavior across each. We develop a systematic pipeline for data construction and knowledge infusion to simulate these fusion scenarios, facilitating a series of controlled experiments. Our investigation reveals that enhancing parametric knowledge within LLMs can significantly bolster their capability for knowledge integration. Nonetheless, we identify persistent challenges in memorizing and eliciting parametric knowledge, and determining parametric knowledge boundaries. Our findings aim to steer future explorations on harmonizing external and parametric knowledge within LLMs. |
| title | Evaluating the External and Parametric Knowledge Fusion of Large Language Models |
| topic | Computation and Language Artificial Intelligence Information Retrieval |
| url | https://arxiv.org/abs/2405.19010 |