Mitigating Knowledge Conflicts in Language Model-Driven Question Answering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910785085636608 |
|---|---|
| author | Cao, Han Zhang, Zhaoyang Li, Xiangtian Wu, Chufan Zhang, Hansong Zhang, Wenqing |
| author_facet | Cao, Han Zhang, Zhaoyang Li, Xiangtian Wu, Chufan Zhang, Hansong Zhang, Wenqing |
| contents | In the context of knowledge-driven seq-to-seq generation tasks, such as document-based question answering and document summarization systems, two fundamental knowledge sources play crucial roles: the inherent knowledge embedded within model parameters and the external knowledge obtained through context. Recent studies revealed a significant challenge: when there exists a misalignment between the model's inherent knowledge and the ground truth answers in training data, the system may exhibit problematic behaviors during inference, such as ignoring input context, or generating unfaithful content. Our investigation proposes a strategy to minimize hallucination by building explicit connection between source inputs and generated outputs. We specifically target a common hallucination pattern in question answering, examining how the correspondence between entities and their contexts during model training influences the system's performance at inference time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11344 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mitigating Knowledge Conflicts in Language Model-Driven Question Answering Cao, Han Zhang, Zhaoyang Li, Xiangtian Wu, Chufan Zhang, Hansong Zhang, Wenqing Computation and Language Artificial Intelligence In the context of knowledge-driven seq-to-seq generation tasks, such as document-based question answering and document summarization systems, two fundamental knowledge sources play crucial roles: the inherent knowledge embedded within model parameters and the external knowledge obtained through context. Recent studies revealed a significant challenge: when there exists a misalignment between the model's inherent knowledge and the ground truth answers in training data, the system may exhibit problematic behaviors during inference, such as ignoring input context, or generating unfaithful content. Our investigation proposes a strategy to minimize hallucination by building explicit connection between source inputs and generated outputs. We specifically target a common hallucination pattern in question answering, examining how the correspondence between entities and their contexts during model training influences the system's performance at inference time. |
| title | Mitigating Knowledge Conflicts in Language Model-Driven Question Answering |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2411.11344 |