MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911245776453632 |
|---|---|
| author | Xia, Shujun Lin, Haokun Wu, Yichen Zhou, Yinan Li, Zixuan Wan, Zhongwei Xing, Xingrun Zheng, Yefeng Li, Xiang Shan, Caifeng Sun, Zhenan Li, Quanzheng |
| author_facet | Xia, Shujun Lin, Haokun Wu, Yichen Zhou, Yinan Li, Zixuan Wan, Zhongwei Xing, Xingrun Zheng, Yefeng Li, Xiang Shan, Caifeng Sun, Zhenan Li, Quanzheng |
| contents | LLMs hold great promise for healthcare applications, but the rapid evolution of medical knowledge and errors in training data often cause them to generate outdated or inaccurate information, limiting their applicability in high-stakes clinical practice. Model editing has emerged as a potential remedy without full retraining. While parameter-based editing often compromises locality and is thus ill-suited for the medical domain, retrieval-based editing offers a more viable alternative. However, it still faces two critical challenges: (1) representation overlap within the medical knowledge space often causes inaccurate retrieval and reduces editing accuracy; (2) existing methods are restricted to single-sample edits, while batch-editing remains largely unexplored despite its importance for real-world medical applications. To address these challenges, we first construct MedVersa, an enhanced benchmark with broader coverage of medical subjects, designed to evaluate both single and batch edits under strict locality constraints. We then propose MedREK, a retrieval-based editing framework that integrates a shared query-key module for precise matching with an attention-based prompt encoder for informative guidance. Experimental results on various medical benchmarks demonstrate that our MedREK achieves superior performance across different core metrics and provides the first validated solution for batch-editing in medical LLMs. Our code and dataset are available at https://github.com/mylittleriver/MedREK. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_13500 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts Xia, Shujun Lin, Haokun Wu, Yichen Zhou, Yinan Li, Zixuan Wan, Zhongwei Xing, Xingrun Zheng, Yefeng Li, Xiang Shan, Caifeng Sun, Zhenan Li, Quanzheng Computation and Language Artificial Intelligence LLMs hold great promise for healthcare applications, but the rapid evolution of medical knowledge and errors in training data often cause them to generate outdated or inaccurate information, limiting their applicability in high-stakes clinical practice. Model editing has emerged as a potential remedy without full retraining. While parameter-based editing often compromises locality and is thus ill-suited for the medical domain, retrieval-based editing offers a more viable alternative. However, it still faces two critical challenges: (1) representation overlap within the medical knowledge space often causes inaccurate retrieval and reduces editing accuracy; (2) existing methods are restricted to single-sample edits, while batch-editing remains largely unexplored despite its importance for real-world medical applications. To address these challenges, we first construct MedVersa, an enhanced benchmark with broader coverage of medical subjects, designed to evaluate both single and batch edits under strict locality constraints. We then propose MedREK, a retrieval-based editing framework that integrates a shared query-key module for precise matching with an attention-based prompt encoder for informative guidance. Experimental results on various medical benchmarks demonstrate that our MedREK achieves superior performance across different core metrics and provides the first validated solution for batch-editing in medical LLMs. Our code and dataset are available at https://github.com/mylittleriver/MedREK. |
| title | MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.13500 |