Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929551733424128 |
|---|---|
| author | Xu, Derong Zhang, Ziheng Zhu, Zhihong Lin, Zhenxi Liu, Qidong Wu, Xian Xu, Tong Zhao, Xiangyu Zheng, Yefeng Chen, Enhong |
| author_facet | Xu, Derong Zhang, Ziheng Zhu, Zhihong Lin, Zhenxi Liu, Qidong Wu, Xian Xu, Tong Zhao, Xiangyu Zheng, Yefeng Chen, Enhong |
| contents | The impressive capabilities of large language models (LLMs) have attracted extensive interests of applying LLMs to medical field. However, the complex nature of clinical environments presents significant hallucination challenges for LLMs, hindering their widespread adoption. In this paper, we address these hallucination issues in the context of Medical Information Extraction (MIE) tasks by introducing ALternate Contrastive Decoding (ALCD). We begin by redefining MIE tasks as an identify-and-classify process. We then separate the identification and classification functions of LLMs by selectively masking the optimization of tokens during fine-tuning. During the inference stage, we alternately contrast output distributions derived from sub-task models. This approach aims to selectively enhance the identification and classification capabilities while minimizing the influence of other inherent abilities in LLMs. Additionally, we propose an alternate adaptive constraint strategy to more effectively adjust the scale and scope of contrastive tokens. Through comprehensive experiments on two different backbones and six diverse medical information extraction tasks, ALCD demonstrates significant improvements in resolving hallucination issues compared to conventional decoding methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_15702 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding Xu, Derong Zhang, Ziheng Zhu, Zhihong Lin, Zhenxi Liu, Qidong Wu, Xian Xu, Tong Zhao, Xiangyu Zheng, Yefeng Chen, Enhong Computation and Language The impressive capabilities of large language models (LLMs) have attracted extensive interests of applying LLMs to medical field. However, the complex nature of clinical environments presents significant hallucination challenges for LLMs, hindering their widespread adoption. In this paper, we address these hallucination issues in the context of Medical Information Extraction (MIE) tasks by introducing ALternate Contrastive Decoding (ALCD). We begin by redefining MIE tasks as an identify-and-classify process. We then separate the identification and classification functions of LLMs by selectively masking the optimization of tokens during fine-tuning. During the inference stage, we alternately contrast output distributions derived from sub-task models. This approach aims to selectively enhance the identification and classification capabilities while minimizing the influence of other inherent abilities in LLMs. Additionally, we propose an alternate adaptive constraint strategy to more effectively adjust the scale and scope of contrastive tokens. Through comprehensive experiments on two different backbones and six diverse medical information extraction tasks, ALCD demonstrates significant improvements in resolving hallucination issues compared to conventional decoding methods. |
| title | Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.15702 |