MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914278961840128 |
|---|---|
| author | Xia, Peng Wang, Jinglu Peng, Yibo Zeng, Kaide Dong, Zihan Wu, Xian Tang, Xiangru Zhu, Hongtu Li, Yun Zhang, Linjun Liu, Shujie Lu, Yan Yao, Huaxiu |
| author_facet | Xia, Peng Wang, Jinglu Peng, Yibo Zeng, Kaide Dong, Zihan Wu, Xian Tang, Xiangru Zhu, Hongtu Li, Yun Zhang, Linjun Liu, Shujie Lu, Yan Yao, Huaxiu |
| contents | Medical Large Vision-Language Models (Med-LVLMs) have shown strong potential in multimodal diagnostic tasks. However, existing single-agent models struggle to generalize across diverse medical specialties, limiting their performance. Recent efforts introduce multi-agent collaboration frameworks inspired by clinical workflows, where general practitioners (GPs) and specialists interact in a fixed sequence. Despite improvements, these static pipelines lack flexibility and adaptability in reasoning. To address this, we propose MMedAgent-RL, a reinforcement learning (RL)-based multi-agent framework that enables dynamic, optimized collaboration among medical agents. Specifically, we train two GP agents based on Qwen2.5-VL via RL: the triage doctor learns to assign patients to appropriate specialties, while the attending physician integrates the judgments from multi-specialists and its own knowledge to make final decisions. To address the inconsistency in specialist outputs, we introduce a curriculum learning (CL)-guided RL strategy with dynamic entropy regulation, progressively teaching the attending physician to balance between imitating specialists and correcting their mistakes. Experiments on five medical VQA benchmarks demonstrate that MMedAgent-RL outperforms both open-source and proprietary Med-LVLMs. Notably, it achieves an average performance gain of 23.6% over strong baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_00555 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoning Xia, Peng Wang, Jinglu Peng, Yibo Zeng, Kaide Dong, Zihan Wu, Xian Tang, Xiangru Zhu, Hongtu Li, Yun Zhang, Linjun Liu, Shujie Lu, Yan Yao, Huaxiu Machine Learning Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Medical Large Vision-Language Models (Med-LVLMs) have shown strong potential in multimodal diagnostic tasks. However, existing single-agent models struggle to generalize across diverse medical specialties, limiting their performance. Recent efforts introduce multi-agent collaboration frameworks inspired by clinical workflows, where general practitioners (GPs) and specialists interact in a fixed sequence. Despite improvements, these static pipelines lack flexibility and adaptability in reasoning. To address this, we propose MMedAgent-RL, a reinforcement learning (RL)-based multi-agent framework that enables dynamic, optimized collaboration among medical agents. Specifically, we train two GP agents based on Qwen2.5-VL via RL: the triage doctor learns to assign patients to appropriate specialties, while the attending physician integrates the judgments from multi-specialists and its own knowledge to make final decisions. To address the inconsistency in specialist outputs, we introduce a curriculum learning (CL)-guided RL strategy with dynamic entropy regulation, progressively teaching the attending physician to balance between imitating specialists and correcting their mistakes. Experiments on five medical VQA benchmarks demonstrate that MMedAgent-RL outperforms both open-source and proprietary Med-LVLMs. Notably, it achieves an average performance gain of 23.6% over strong baselines. |
| title | MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoning |
| topic | Machine Learning Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.00555 |