LLM, Reporting In! Medical Information Extraction Across Prompting, Fine-tuning and Post-correction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911191220092928 |
|---|---|
| author | Belmadani, Ikram Hashemi, Parisa Nazari Sebbag, Thomas Favre, Benoit Fortier, Guillaume Quiniou, Solen Morin, Emmanuel Dufour, Richard |
| author_facet | Belmadani, Ikram Hashemi, Parisa Nazari Sebbag, Thomas Favre, Benoit Fortier, Guillaume Quiniou, Solen Morin, Emmanuel Dufour, Richard |
| contents | This work presents our participation in the EvalLLM 2025 challenge on biomedical Named Entity Recognition (NER) and health event extraction in French (few-shot setting). For NER, we propose three approaches combining large language models (LLMs), annotation guidelines, synthetic data, and post-processing: (1) in-context learning (ICL) with GPT-4.1, incorporating automatic selection of 10 examples and a summary of the annotation guidelines into the prompt, (2) the universal NER system GLiNER, fine-tuned on a synthetic corpus and then verified by an LLM in post-processing, and (3) the open LLM LLaMA-3.1-8B-Instruct, fine-tuned on the same synthetic corpus. Event extraction uses the same ICL strategy with GPT-4.1, reusing the guideline summary in the prompt. Results show GPT-4.1 leads with a macro-F1 of 61.53% for NER and 15.02% for event extraction, highlighting the importance of well-crafted prompting to maximize performance in very low-resource scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_03577 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLM, Reporting In! Medical Information Extraction Across Prompting, Fine-tuning and Post-correction Belmadani, Ikram Hashemi, Parisa Nazari Sebbag, Thomas Favre, Benoit Fortier, Guillaume Quiniou, Solen Morin, Emmanuel Dufour, Richard Computation and Language Information Retrieval This work presents our participation in the EvalLLM 2025 challenge on biomedical Named Entity Recognition (NER) and health event extraction in French (few-shot setting). For NER, we propose three approaches combining large language models (LLMs), annotation guidelines, synthetic data, and post-processing: (1) in-context learning (ICL) with GPT-4.1, incorporating automatic selection of 10 examples and a summary of the annotation guidelines into the prompt, (2) the universal NER system GLiNER, fine-tuned on a synthetic corpus and then verified by an LLM in post-processing, and (3) the open LLM LLaMA-3.1-8B-Instruct, fine-tuned on the same synthetic corpus. Event extraction uses the same ICL strategy with GPT-4.1, reusing the guideline summary in the prompt. Results show GPT-4.1 leads with a macro-F1 of 61.53% for NER and 15.02% for event extraction, highlighting the importance of well-crafted prompting to maximize performance in very low-resource scenarios. |
| title | LLM, Reporting In! Medical Information Extraction Across Prompting, Fine-tuning and Post-correction |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2510.03577 |