RadVLM: A Multitask Conversational Vision-Language Model for Radiology
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917000343715840 |
|---|---|
| author | Deperrois, Nicolas Matsuo, Hidetoshi Ruipérez-Campillo, Samuel Vandenhirtz, Moritz Laguna, Sonia Ryser, Alain Fujimoto, Koji Nishio, Mizuho Sutter, Thomas M. Vogt, Julia E. Kluckert, Jonas Frauenfelder, Thomas Blüthgen, Christian Nooralahzadeh, Farhad Krauthammer, Michael |
| author_facet | Deperrois, Nicolas Matsuo, Hidetoshi Ruipérez-Campillo, Samuel Vandenhirtz, Moritz Laguna, Sonia Ryser, Alain Fujimoto, Koji Nishio, Mizuho Sutter, Thomas M. Vogt, Julia E. Kluckert, Jonas Frauenfelder, Thomas Blüthgen, Christian Nooralahzadeh, Farhad Krauthammer, Michael |
| contents | The widespread use of chest X-rays (CXRs), coupled with a shortage of radiologists, has driven growing interest in automated CXR analysis and AI-assisted reporting. While existing vision-language models (VLMs) show promise in specific tasks such as report generation or abnormality detection, they often lack support for interactive diagnostic capabilities. In this work we present RadVLM, a compact, multitask conversational foundation model designed for CXR interpretation. To this end, we curate a large-scale instruction dataset comprising over 1 million image-instruction pairs containing both single-turn tasks -- such as report generation, abnormality classification, and visual grounding -- and multi-turn, multi-task conversational interactions. After fine-tuning RadVLM on this instruction dataset, we evaluate it across different tasks along with re-implemented baseline VLMs. Our results show that RadVLM achieves state-of-the-art performance in conversational capabilities and visual grounding while remaining competitive in other radiology tasks. Ablation studies further highlight the benefit of joint training across multiple tasks, particularly for scenarios with limited annotated data. Together, these findings highlight the potential of RadVLM as a clinically relevant AI assistant, providing structured CXR interpretation and conversational capabilities to support more effective and accessible diagnostic workflows. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_03333 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RadVLM: A Multitask Conversational Vision-Language Model for Radiology Deperrois, Nicolas Matsuo, Hidetoshi Ruipérez-Campillo, Samuel Vandenhirtz, Moritz Laguna, Sonia Ryser, Alain Fujimoto, Koji Nishio, Mizuho Sutter, Thomas M. Vogt, Julia E. Kluckert, Jonas Frauenfelder, Thomas Blüthgen, Christian Nooralahzadeh, Farhad Krauthammer, Michael Computer Vision and Pattern Recognition Artificial Intelligence The widespread use of chest X-rays (CXRs), coupled with a shortage of radiologists, has driven growing interest in automated CXR analysis and AI-assisted reporting. While existing vision-language models (VLMs) show promise in specific tasks such as report generation or abnormality detection, they often lack support for interactive diagnostic capabilities. In this work we present RadVLM, a compact, multitask conversational foundation model designed for CXR interpretation. To this end, we curate a large-scale instruction dataset comprising over 1 million image-instruction pairs containing both single-turn tasks -- such as report generation, abnormality classification, and visual grounding -- and multi-turn, multi-task conversational interactions. After fine-tuning RadVLM on this instruction dataset, we evaluate it across different tasks along with re-implemented baseline VLMs. Our results show that RadVLM achieves state-of-the-art performance in conversational capabilities and visual grounding while remaining competitive in other radiology tasks. Ablation studies further highlight the benefit of joint training across multiple tasks, particularly for scenarios with limited annotated data. Together, these findings highlight the potential of RadVLM as a clinically relevant AI assistant, providing structured CXR interpretation and conversational capabilities to support more effective and accessible diagnostic workflows. |
| title | RadVLM: A Multitask Conversational Vision-Language Model for Radiology |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2502.03333 |