Multi-Resolution Pathology-Language Pre-training Model with Text-Guided Visual Representation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913809971544064 |
|---|---|
| author | Albastaki, Shahad Sohail, Anabia Ganapathi, Iyyakutti Iyappan Alawode, Basit Khan, Asim Javed, Sajid Werghi, Naoufel Bennamoun, Mohammed Mahmood, Arif |
| author_facet | Albastaki, Shahad Sohail, Anabia Ganapathi, Iyyakutti Iyappan Alawode, Basit Khan, Asim Javed, Sajid Werghi, Naoufel Bennamoun, Mohammed Mahmood, Arif |
| contents | In Computational Pathology (CPath), the introduction of Vision-Language Models (VLMs) has opened new avenues for research, focusing primarily on aligning image-text pairs at a single magnification level. However, this approach might not be sufficient for tasks like cancer subtype classification, tissue phenotyping, and survival analysis due to the limited level of detail that a single-resolution image can provide. Addressing this, we propose a novel multi-resolution paradigm leveraging Whole Slide Images (WSIs) to extract histology patches at multiple resolutions and generate corresponding textual descriptions through advanced CPath VLM. We introduce visual-textual alignment at multiple resolutions as well as cross-resolution alignment to establish more effective text-guided visual representations. Cross-resolution alignment using a multimodal encoder enhances the model's ability to capture context from multiple resolutions in histology images. Our model aims to capture a broader range of information, supported by novel loss functions, enriches feature representation, improves discriminative ability, and enhances generalization across different resolutions. Pre-trained on a comprehensive TCGA dataset with 34 million image-language pairs at various resolutions, our fine-tuned model outperforms state-of-the-art (SOTA) counterparts across multiple datasets and tasks, demonstrating its effectiveness in CPath. The code is available on GitHub at: https://github.com/BasitAlawode/MR-PLIP |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_18856 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Multi-Resolution Pathology-Language Pre-training Model with Text-Guided Visual Representation Albastaki, Shahad Sohail, Anabia Ganapathi, Iyyakutti Iyappan Alawode, Basit Khan, Asim Javed, Sajid Werghi, Naoufel Bennamoun, Mohammed Mahmood, Arif Computer Vision and Pattern Recognition In Computational Pathology (CPath), the introduction of Vision-Language Models (VLMs) has opened new avenues for research, focusing primarily on aligning image-text pairs at a single magnification level. However, this approach might not be sufficient for tasks like cancer subtype classification, tissue phenotyping, and survival analysis due to the limited level of detail that a single-resolution image can provide. Addressing this, we propose a novel multi-resolution paradigm leveraging Whole Slide Images (WSIs) to extract histology patches at multiple resolutions and generate corresponding textual descriptions through advanced CPath VLM. We introduce visual-textual alignment at multiple resolutions as well as cross-resolution alignment to establish more effective text-guided visual representations. Cross-resolution alignment using a multimodal encoder enhances the model's ability to capture context from multiple resolutions in histology images. Our model aims to capture a broader range of information, supported by novel loss functions, enriches feature representation, improves discriminative ability, and enhances generalization across different resolutions. Pre-trained on a comprehensive TCGA dataset with 34 million image-language pairs at various resolutions, our fine-tuned model outperforms state-of-the-art (SOTA) counterparts across multiple datasets and tasks, demonstrating its effectiveness in CPath. The code is available on GitHub at: https://github.com/BasitAlawode/MR-PLIP |
| title | Multi-Resolution Pathology-Language Pre-training Model with Text-Guided Visual Representation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.18856 |