Improving LLM's Attachment to External Knowledge In Dialogue Generation Tasks Through Entity Anonymization
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911266681913344 |
|---|---|
| author | Sheikhi, Hadi Huang, Chenyang Zaïane, Osmar R. |
| author_facet | Sheikhi, Hadi Huang, Chenyang Zaïane, Osmar R. |
| contents | Knowledge graph-based dialogue generation (KG-DG) is a challenging task requiring models to effectively incorporate external knowledge into conversational responses. While large language models (LLMs) have achieved impressive results across various NLP tasks, their ability to utilize external knowledge in KG-DG remains under-explored. We observe that LLMs often rely on internal knowledge, leading to detachment from provided knowledge graphs, even when they are given a flawlessly retrieved knowledge graph. First, we introduce LLM-KAT, an evaluation procedure for measuring knowledge attachment in generated responses. Second, we propose a simple yet effective entity anonymization technique to encourage LLMs to better leverage external knowledge. Experiments on the OpenDialKG dataset demonstrate that our approach improves LLMs' attachment on external knowledge. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_11946 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Improving LLM's Attachment to External Knowledge In Dialogue Generation Tasks Through Entity Anonymization Sheikhi, Hadi Huang, Chenyang Zaïane, Osmar R. Computation and Language Machine Learning Knowledge graph-based dialogue generation (KG-DG) is a challenging task requiring models to effectively incorporate external knowledge into conversational responses. While large language models (LLMs) have achieved impressive results across various NLP tasks, their ability to utilize external knowledge in KG-DG remains under-explored. We observe that LLMs often rely on internal knowledge, leading to detachment from provided knowledge graphs, even when they are given a flawlessly retrieved knowledge graph. First, we introduce LLM-KAT, an evaluation procedure for measuring knowledge attachment in generated responses. Second, we propose a simple yet effective entity anonymization technique to encourage LLMs to better leverage external knowledge. Experiments on the OpenDialKG dataset demonstrate that our approach improves LLMs' attachment on external knowledge. |
| title | Improving LLM's Attachment to External Knowledge In Dialogue Generation Tasks Through Entity Anonymization |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2511.11946 |