ReXInTheWild: A Unified Benchmark for Medical Photograph Understanding
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911530918871040 |
|---|---|
| author | Banerjee, Oishi Kim, Sung Eun Willauer, Alexandra N. Kernbach, Julius M. Alomaish, Abeer Rihan Alghamdi, Reema Abdulwahab S. Alomaish, Hassan Rayhan Baharoon, Mohammed Zhang, Xiaoman Acosta, Julian Nicolas Zhou, Christine Rajpurkar, Pranav |
| author_facet | Banerjee, Oishi Kim, Sung Eun Willauer, Alexandra N. Kernbach, Julius M. Alomaish, Abeer Rihan Alghamdi, Reema Abdulwahab S. Alomaish, Hassan Rayhan Baharoon, Mohammed Zhang, Xiaoman Acosta, Julian Nicolas Zhou, Christine Rajpurkar, Pranav |
| contents | Everyday photographs taken with ordinary cameras are already widely used in telemedicine and other online health conversations, yet no comprehensive benchmark evaluates whether vision-language models can interpret their medical content. Analyzing these images requires both fine-grained natural image understanding and domain-specific medical reasoning, a combination that challenges both general-purpose and specialized models. We introduce ReXInTheWild, a benchmark of 955 clinician-verified multiple-choice questions spanning seven clinical topics across 484 photographs sourced from the biomedical literature. When evaluated on ReXInTheWild, leading multimodal large language models show substantial performance variation: Gemini-3 achieves 78% accuracy, followed by Claude Opus 4.5 (72%) and GPT-5 (68%), while the medical specialist model MedGemma reaches only 37%. A systematic error analysis also reveals four categories of common errors, ranging from low-level geometric errors to high-level reasoning failures and requiring different mitigation strategies. ReXInTheWild provides a challenging, clinically grounded benchmark at the intersection of natural image understanding and medical reasoning. The dataset is available on HuggingFace. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_19517 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ReXInTheWild: A Unified Benchmark for Medical Photograph Understanding Banerjee, Oishi Kim, Sung Eun Willauer, Alexandra N. Kernbach, Julius M. Alomaish, Abeer Rihan Alghamdi, Reema Abdulwahab S. Alomaish, Hassan Rayhan Baharoon, Mohammed Zhang, Xiaoman Acosta, Julian Nicolas Zhou, Christine Rajpurkar, Pranav Computer Vision and Pattern Recognition Machine Learning Everyday photographs taken with ordinary cameras are already widely used in telemedicine and other online health conversations, yet no comprehensive benchmark evaluates whether vision-language models can interpret their medical content. Analyzing these images requires both fine-grained natural image understanding and domain-specific medical reasoning, a combination that challenges both general-purpose and specialized models. We introduce ReXInTheWild, a benchmark of 955 clinician-verified multiple-choice questions spanning seven clinical topics across 484 photographs sourced from the biomedical literature. When evaluated on ReXInTheWild, leading multimodal large language models show substantial performance variation: Gemini-3 achieves 78% accuracy, followed by Claude Opus 4.5 (72%) and GPT-5 (68%), while the medical specialist model MedGemma reaches only 37%. A systematic error analysis also reveals four categories of common errors, ranging from low-level geometric errors to high-level reasoning failures and requiring different mitigation strategies. ReXInTheWild provides a challenging, clinically grounded benchmark at the intersection of natural image understanding and medical reasoning. The dataset is available on HuggingFace. |
| title | ReXInTheWild: A Unified Benchmark for Medical Photograph Understanding |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2603.19517 |