Integrating MedCLIP and Cross-Modal Fusion for Automatic Radiology Report Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913604276584448 |
|---|---|
| author | Han, Qianhao Liu, Junyi Qin, Zengchang Zheng, Zheng |
| author_facet | Han, Qianhao Liu, Junyi Qin, Zengchang Zheng, Zheng |
| contents | Automating radiology report generation can significantly reduce the workload of radiologists and enhance the accuracy, consistency, and efficiency of clinical documentation.We propose a novel cross-modal framework that uses MedCLIP as both a vision extractor and a retrieval mechanism to improve the process of medical report generation.By extracting retrieved report features and image features through an attention-based extract module, and integrating them with a fusion module, our method improves the coherence and clinical relevance of generated reports.Experimental results on the widely used IU-Xray dataset demonstrate the effectiveness of our approach, showing improvements over commonly used methods in both report quality and relevance.Additionally, ablation studies provide further validation of the framework, highlighting the importance of accurate report retrieval and feature integration in generating comprehensive medical reports. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_07141 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Integrating MedCLIP and Cross-Modal Fusion for Automatic Radiology Report Generation Han, Qianhao Liu, Junyi Qin, Zengchang Zheng, Zheng Computer Vision and Pattern Recognition Automating radiology report generation can significantly reduce the workload of radiologists and enhance the accuracy, consistency, and efficiency of clinical documentation.We propose a novel cross-modal framework that uses MedCLIP as both a vision extractor and a retrieval mechanism to improve the process of medical report generation.By extracting retrieved report features and image features through an attention-based extract module, and integrating them with a fusion module, our method improves the coherence and clinical relevance of generated reports.Experimental results on the widely used IU-Xray dataset demonstrate the effectiveness of our approach, showing improvements over commonly used methods in both report quality and relevance.Additionally, ablation studies provide further validation of the framework, highlighting the importance of accurate report retrieval and feature integration in generating comprehensive medical reports. |
| title | Integrating MedCLIP and Cross-Modal Fusion for Automatic Radiology Report Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.07141 |