Enhancing Representation in Medical Vision-Language Foundation Models via Multi-Scale Information Extraction Techniques
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929255748730880 |
|---|---|
| author | Huang, Weijian Li, Cheng Zhou, Hong-Yu Liu, Jiarun Yang, Hao Liang, Yong Shi, Guangming Zheng, Hairong Wang, Shanshan |
| author_facet | Huang, Weijian Li, Cheng Zhou, Hong-Yu Liu, Jiarun Yang, Hao Liang, Yong Shi, Guangming Zheng, Hairong Wang, Shanshan |
| contents | The development of medical vision-language foundation models has attracted significant attention in the field of medicine and healthcare due to their promising prospect in various clinical applications. While previous studies have commonly focused on feature learning at a single learning scale, investigation on integrating multi-scale information is lacking, which may hinder the potential for mutual reinforcement among these features. This paper aims to bridge this gap by proposing a method that effectively exploits multi-scale information to enhance the performance of medical foundation models. The proposed method simultaneously exploits features at the local, instance, modality and global aspects, facilitating comprehensive representation learning within the models. We evaluate the effectiveness of the proposed method on six open-source datasets across different clinical tasks, demonstrating its ability to enhance the performance of medical foundation models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_01583 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Enhancing Representation in Medical Vision-Language Foundation Models via Multi-Scale Information Extraction Techniques Huang, Weijian Li, Cheng Zhou, Hong-Yu Liu, Jiarun Yang, Hao Liang, Yong Shi, Guangming Zheng, Hairong Wang, Shanshan Computer Vision and Pattern Recognition The development of medical vision-language foundation models has attracted significant attention in the field of medicine and healthcare due to their promising prospect in various clinical applications. While previous studies have commonly focused on feature learning at a single learning scale, investigation on integrating multi-scale information is lacking, which may hinder the potential for mutual reinforcement among these features. This paper aims to bridge this gap by proposing a method that effectively exploits multi-scale information to enhance the performance of medical foundation models. The proposed method simultaneously exploits features at the local, instance, modality and global aspects, facilitating comprehensive representation learning within the models. We evaluate the effectiveness of the proposed method on six open-source datasets across different clinical tasks, demonstrating its ability to enhance the performance of medical foundation models. |
| title | Enhancing Representation in Medical Vision-Language Foundation Models via Multi-Scale Information Extraction Techniques |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.01583 |