Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.06581 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913014595190784 |
|---|---|
| author | Su, Yuhao Choudhuri, Anwesa Gao, Zhongpai Planche, Benjamin Nguyen, Van Nguyen Zheng, Meng Shen, Yuhan Innanje, Arun Chen, Terrence Elhamifar, Ehsan Wu, Ziyan |
| author_facet | Su, Yuhao Choudhuri, Anwesa Gao, Zhongpai Planche, Benjamin Nguyen, Van Nguyen Zheng, Meng Shen, Yuhan Innanje, Arun Chen, Terrence Elhamifar, Ehsan Wu, Ziyan |
| contents | Large vision-language models struggle with medical video understanding, where spatial precision, temporal reasoning, and clinical semantics are critical. To address this, we first introduce \textbf{MedVidBench}, a large-scale benchmark of 531,850 video-instruction pairs across 8 medical sources spanning video, segment, and frame-level tasks, curated through a rigorous quality assurance pipeline with expert-guided prompting and dual-model validation. While supervised fine-tuning on MedVidBench yields noticeable gains, standard Reinforcement Learning (RL) fails due to imbalanced reward scales across datasets, which destabilizes optimization and leads to training collapse. To overcome this, we introduce \textbf{MedGRPO}, a novel RL framework for balanced multi-dataset training with two key innovations: (1) \emph{cross-dataset reward normalization} that maps each dataset's median performance to a common reward value, ensuring fair optimization regardless of difficulty, and (2) a \emph{medical LLM judge} that evaluates caption quality on five clinical dimensions through comparative similarity scoring. Supervised fine-tuning Qwen2.5-VL-7B on MedVidBench outperforms GPT-4.1 and Gemini-2.5-Flash across all tasks, while MedGRPO further improves the SFT baseline on grounding and captioning. Our work establishes a foundational benchmark and training methodology for advancing medical video understanding with VLMs. Our project website is available at: https://uii-america.github.io/MedGRPO/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_06581 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MedGRPO: Multi-Task Reinforcement Learning for Heterogeneous Medical Video Understanding Su, Yuhao Choudhuri, Anwesa Gao, Zhongpai Planche, Benjamin Nguyen, Van Nguyen Zheng, Meng Shen, Yuhan Innanje, Arun Chen, Terrence Elhamifar, Ehsan Wu, Ziyan Computer Vision and Pattern Recognition Large vision-language models struggle with medical video understanding, where spatial precision, temporal reasoning, and clinical semantics are critical. To address this, we first introduce \textbf{MedVidBench}, a large-scale benchmark of 531,850 video-instruction pairs across 8 medical sources spanning video, segment, and frame-level tasks, curated through a rigorous quality assurance pipeline with expert-guided prompting and dual-model validation. While supervised fine-tuning on MedVidBench yields noticeable gains, standard Reinforcement Learning (RL) fails due to imbalanced reward scales across datasets, which destabilizes optimization and leads to training collapse. To overcome this, we introduce \textbf{MedGRPO}, a novel RL framework for balanced multi-dataset training with two key innovations: (1) \emph{cross-dataset reward normalization} that maps each dataset's median performance to a common reward value, ensuring fair optimization regardless of difficulty, and (2) a \emph{medical LLM judge} that evaluates caption quality on five clinical dimensions through comparative similarity scoring. Supervised fine-tuning Qwen2.5-VL-7B on MedVidBench outperforms GPT-4.1 and Gemini-2.5-Flash across all tasks, while MedGRPO further improves the SFT baseline on grounding and captioning. Our work establishes a foundational benchmark and training methodology for advancing medical video understanding with VLMs. Our project website is available at: https://uii-america.github.io/MedGRPO/. |
| title | MedGRPO: Multi-Task Reinforcement Learning for Heterogeneous Medical Video Understanding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.06581 |