LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909346857746432 |
|---|---|
| author | Li, Jiachun Cao, Pengfei Wang, Chenhao Jin, Zhuoran Chen, Yubo Liu, Kang Jiang, Xiaojian Xu, Jiexin Zhao, Jun |
| author_facet | Li, Jiachun Cao, Pengfei Wang, Chenhao Jin, Zhuoran Chen, Yubo Liu, Kang Jiang, Xiaojian Xu, Jiexin Zhao, Jun |
| contents | Large language models (LLMs) sometimes demonstrate poor performance on knowledge-intensive tasks, commonsense reasoning is one of them. Researchers typically address these issues by retrieving related knowledge from knowledge graphs or employing self-enhancement methods to elicit knowledge in LLMs. However, noisy knowledge and invalid reasoning issues hamper their ability to answer questions accurately. To this end, we propose a novel method named eliciting, filtering and integrating knowledge in large language model (LINKED). In it, we design a reward model to filter out the noisy knowledge and take the marginal consistent reasoning module to reduce invalid reasoning. With our comprehensive experiments on two complex commonsense reasoning benchmarks, our method outperforms SOTA baselines (up to 9.0% improvement of accuracy). Besides, to measure the positive and negative impact of the injected knowledge, we propose a new metric called effectiveness-preservation score for the knowledge enhancement works. Finally, through extensive experiments, we conduct an in-depth analysis and find many meaningful conclusions about LLMs in commonsense reasoning tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_09541 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning Li, Jiachun Cao, Pengfei Wang, Chenhao Jin, Zhuoran Chen, Yubo Liu, Kang Jiang, Xiaojian Xu, Jiexin Zhao, Jun Computation and Language Artificial Intelligence Large language models (LLMs) sometimes demonstrate poor performance on knowledge-intensive tasks, commonsense reasoning is one of them. Researchers typically address these issues by retrieving related knowledge from knowledge graphs or employing self-enhancement methods to elicit knowledge in LLMs. However, noisy knowledge and invalid reasoning issues hamper their ability to answer questions accurately. To this end, we propose a novel method named eliciting, filtering and integrating knowledge in large language model (LINKED). In it, we design a reward model to filter out the noisy knowledge and take the marginal consistent reasoning module to reduce invalid reasoning. With our comprehensive experiments on two complex commonsense reasoning benchmarks, our method outperforms SOTA baselines (up to 9.0% improvement of accuracy). Besides, to measure the positive and negative impact of the injected knowledge, we propose a new metric called effectiveness-preservation score for the knowledge enhancement works. Finally, through extensive experiments, we conduct an in-depth analysis and find many meaningful conclusions about LLMs in commonsense reasoning tasks. |
| title | LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2410.09541 |