Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.14306 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910160829546496 |
|---|---|
| author | Causio, Francesco Andrea De Vita, Vittorio Riccomi, Olivia Ferramola, Michele Felizzi, Federico Tosi, Alessandro Cristiano, Antonio De Mori, Lorenzo Battipaglia, Chiara Sawaya, Melissa De Angelis, Luigi Di Pumpo, Marcello Piscitelli, Alessandra Risuleo, Pietro Eric Longo, Alessia Vojvodic, Giulia Vassalli, Mariapia Castaniti, Bianca Destro Scarsi, Nicolò Del Medico, Manuel |
| author_facet | Causio, Francesco Andrea De Vita, Vittorio Riccomi, Olivia Ferramola, Michele Felizzi, Federico Tosi, Alessandro Cristiano, Antonio De Mori, Lorenzo Battipaglia, Chiara Sawaya, Melissa De Angelis, Luigi Di Pumpo, Marcello Piscitelli, Alessandra Risuleo, Pietro Eric Longo, Alessia Vojvodic, Giulia Vassalli, Mariapia Castaniti, Bianca Destro Scarsi, Nicolò Del Medico, Manuel |
| contents | While Large Language Models (LLMs) have demonstrated high proficiency on English-centric medical examinations, their performance often declines when faced with non-English languages and multimodal diagnostic tasks. This study protocol describes the development of EuropeMedQA, the first comprehensive, multilingual, and multimodal medical examination dataset sourced from official regulatory exams in Italy, France, Spain, and Portugal. Following FAIR data principles and SPIRIT-AI guidelines, we describe a rigorous curation process and an automated translation pipeline for comparative analysis. We evaluate contemporary multimodal LLMs using a zero-shot, strictly constrained prompting strategy to assess cross-lingual transfer and visual reasoning. EuropeMedQA aims to provide a contamination-resistant benchmark that reflects the complexity of European clinical practices and fosters the development of more generalizable medical AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_14306 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | EuropeMedQA Study Protocol: A Multilingual, Multimodal Medical Examination Dataset for Language Model Evaluation Causio, Francesco Andrea De Vita, Vittorio Riccomi, Olivia Ferramola, Michele Felizzi, Federico Tosi, Alessandro Cristiano, Antonio De Mori, Lorenzo Battipaglia, Chiara Sawaya, Melissa De Angelis, Luigi Di Pumpo, Marcello Piscitelli, Alessandra Risuleo, Pietro Eric Longo, Alessia Vojvodic, Giulia Vassalli, Mariapia Castaniti, Bianca Destro Scarsi, Nicolò Del Medico, Manuel Computation and Language Artificial Intelligence While Large Language Models (LLMs) have demonstrated high proficiency on English-centric medical examinations, their performance often declines when faced with non-English languages and multimodal diagnostic tasks. This study protocol describes the development of EuropeMedQA, the first comprehensive, multilingual, and multimodal medical examination dataset sourced from official regulatory exams in Italy, France, Spain, and Portugal. Following FAIR data principles and SPIRIT-AI guidelines, we describe a rigorous curation process and an automated translation pipeline for comparative analysis. We evaluate contemporary multimodal LLMs using a zero-shot, strictly constrained prompting strategy to assess cross-lingual transfer and visual reasoning. EuropeMedQA aims to provide a contamination-resistant benchmark that reflects the complexity of European clinical practices and fosters the development of more generalizable medical AI. |
| title | EuropeMedQA Study Protocol: A Multilingual, Multimodal Medical Examination Dataset for Language Model Evaluation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2604.14306 |