RankByGene: Gene-Guided Histopathology Representation Learning Through Cross-Modal Ranking Consistency
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911740014362624 |
|---|---|
| author | Huang, Wentao Xu, Meilong Hu, Xiaoling Abousamra, Shahira Ganguly, Aniruddha Kapse, Saarthak Yurovsky, Alisa Prasanna, Prateek Kurc, Tahsin Saltz, Joel Miller, Michael L. Chen, Chao |
| author_facet | Huang, Wentao Xu, Meilong Hu, Xiaoling Abousamra, Shahira Ganguly, Aniruddha Kapse, Saarthak Yurovsky, Alisa Prasanna, Prateek Kurc, Tahsin Saltz, Joel Miller, Michael L. Chen, Chao |
| contents | Spatial transcriptomics (ST) provides essential spatial context by mapping gene expression within tissue, enabling detailed study of cellular heterogeneity and tissue organization. However, aligning ST data with histology images poses challenges due to inherent spatial distortions and modality-specific variations. Existing methods largely rely on direct alignment, which often fails to capture complex cross-modal relationships. To address these limitations, we propose a novel framework that aligns gene and image features using a ranking-based alignment loss, preserving relative similarity across modalities and enabling robust multi-scale alignment. To further enhance the alignment's stability, we employ self-supervised knowledge distillation with a teacher-student network architecture, effectively mitigating disruptions from high dimensionality, sparsity, and noise in gene expression data. Extensive experiments on seven public datasets that encompass gene expression prediction, slide-level classification, and survival analysis demonstrate the efficacy of our method, showing improved alignment and predictive performance over existing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_15076 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RankByGene: Gene-Guided Histopathology Representation Learning Through Cross-Modal Ranking Consistency Huang, Wentao Xu, Meilong Hu, Xiaoling Abousamra, Shahira Ganguly, Aniruddha Kapse, Saarthak Yurovsky, Alisa Prasanna, Prateek Kurc, Tahsin Saltz, Joel Miller, Michael L. Chen, Chao Image and Video Processing Computer Vision and Pattern Recognition Quantitative Methods Spatial transcriptomics (ST) provides essential spatial context by mapping gene expression within tissue, enabling detailed study of cellular heterogeneity and tissue organization. However, aligning ST data with histology images poses challenges due to inherent spatial distortions and modality-specific variations. Existing methods largely rely on direct alignment, which often fails to capture complex cross-modal relationships. To address these limitations, we propose a novel framework that aligns gene and image features using a ranking-based alignment loss, preserving relative similarity across modalities and enabling robust multi-scale alignment. To further enhance the alignment's stability, we employ self-supervised knowledge distillation with a teacher-student network architecture, effectively mitigating disruptions from high dimensionality, sparsity, and noise in gene expression data. Extensive experiments on seven public datasets that encompass gene expression prediction, slide-level classification, and survival analysis demonstrate the efficacy of our method, showing improved alignment and predictive performance over existing methods. |
| title | RankByGene: Gene-Guided Histopathology Representation Learning Through Cross-Modal Ranking Consistency |
| topic | Image and Video Processing Computer Vision and Pattern Recognition Quantitative Methods |
| url | https://arxiv.org/abs/2411.15076 |