Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2510.12482 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908592150413312 |
|---|---|
| author | Chai, Shurong JAIN, Rahul Kumar Xu, Rui Mo, Shaocong Hou, Ruibo Teng, Shiyu Liu, Jiaqing Lin, Lanfen Chen, Yen-Wei |
| author_facet | Chai, Shurong JAIN, Rahul Kumar Xu, Rui Mo, Shaocong Hou, Ruibo Teng, Shiyu Liu, Jiaqing Lin, Lanfen Chen, Yen-Wei |
| contents | Deep learning relies heavily on data augmentation to mitigate limited data, especially in medical imaging. Recent multimodal learning integrates text and images for segmentation, known as referring or text-guided image segmentation. However, common augmentations like rotation and flipping disrupt spatial alignment between image and text, weakening performance. To address this, we propose an early fusion framework that combines text and visual features before augmentation, preserving spatial consistency. We also design a lightweight generator that projects text embeddings into visual space, bridging semantic gaps. Visualization of generated pseudo-images shows accurate region localization. Our method is evaluated on three medical imaging tasks and four segmentation frameworks, achieving state-of-the-art results. Code is publicly available on GitHub: https://github.com/11yxk/MedSeg_EarlyFusion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_12482 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Text-Image Fusion Method with Data Augmentation Capabilities for Referring Medical Image Segmentation Chai, Shurong JAIN, Rahul Kumar Xu, Rui Mo, Shaocong Hou, Ruibo Teng, Shiyu Liu, Jiaqing Lin, Lanfen Chen, Yen-Wei Computer Vision and Pattern Recognition Artificial Intelligence Deep learning relies heavily on data augmentation to mitigate limited data, especially in medical imaging. Recent multimodal learning integrates text and images for segmentation, known as referring or text-guided image segmentation. However, common augmentations like rotation and flipping disrupt spatial alignment between image and text, weakening performance. To address this, we propose an early fusion framework that combines text and visual features before augmentation, preserving spatial consistency. We also design a lightweight generator that projects text embeddings into visual space, bridging semantic gaps. Visualization of generated pseudo-images shows accurate region localization. Our method is evaluated on three medical imaging tasks and four segmentation frameworks, achieving state-of-the-art results. Code is publicly available on GitHub: https://github.com/11yxk/MedSeg_EarlyFusion. |
| title | A Text-Image Fusion Method with Data Augmentation Capabilities for Referring Medical Image Segmentation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2510.12482 |