HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916624814047232 |
|---|---|
| author | Lin, Tianwei Zhang, Wenqiao Li, Sijing Yuan, Yuqian Yu, Binhe Li, Haoyuan He, Wanggui Jiang, Hao Li, Mengze Song, Xiaohui Tang, Siliang Xiao, Jun Lin, Hui Zhuang, Yueting Ooi, Beng Chin |
| author_facet | Lin, Tianwei Zhang, Wenqiao Li, Sijing Yuan, Yuqian Yu, Binhe Li, Haoyuan He, Wanggui Jiang, Hao Li, Mengze Song, Xiaohui Tang, Siliang Xiao, Jun Lin, Hui Zhuang, Yueting Ooi, Beng Chin |
| contents | We present HealthGPT, a powerful Medical Large Vision-Language Model (Med-LVLM) that integrates medical visual comprehension and generation capabilities within a unified autoregressive paradigm. Our bootstrapping philosophy is to progressively adapt heterogeneous comprehension and generation knowledge to pre-trained large language models (LLMs). This is achieved through a novel heterogeneous low-rank adaptation (H-LoRA) technique, which is complemented by a tailored hierarchical visual perception approach and a three-stage learning strategy. To effectively learn the HealthGPT, we devise a comprehensive medical domain-specific comprehension and generation dataset called VL-Health. Experimental results demonstrate exceptional performance and scalability of HealthGPT in medical visual unified tasks. Our project can be accessed at https://github.com/DCDmllm/HealthGPT. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_09838 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation Lin, Tianwei Zhang, Wenqiao Li, Sijing Yuan, Yuqian Yu, Binhe Li, Haoyuan He, Wanggui Jiang, Hao Li, Mengze Song, Xiaohui Tang, Siliang Xiao, Jun Lin, Hui Zhuang, Yueting Ooi, Beng Chin Computer Vision and Pattern Recognition Artificial Intelligence We present HealthGPT, a powerful Medical Large Vision-Language Model (Med-LVLM) that integrates medical visual comprehension and generation capabilities within a unified autoregressive paradigm. Our bootstrapping philosophy is to progressively adapt heterogeneous comprehension and generation knowledge to pre-trained large language models (LLMs). This is achieved through a novel heterogeneous low-rank adaptation (H-LoRA) technique, which is complemented by a tailored hierarchical visual perception approach and a three-stage learning strategy. To effectively learn the HealthGPT, we devise a comprehensive medical domain-specific comprehension and generation dataset called VL-Health. Experimental results demonstrate exceptional performance and scalability of HealthGPT in medical visual unified tasks. Our project can be accessed at https://github.com/DCDmllm/HealthGPT. |
| title | HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2502.09838 |