A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908936733458432 |
|---|---|
| author | Xiang, Jinxi Hou, Siyu Li, Yuchen Quinton, Ryan Zhang, Xiaoming Eweje, Feyisope Luo, Xiangde Chen, Yijiang Li, Zhe Bergstrom, Colin Kim, Ted Willens, Sierra Olguin, Francesca Maria Abikenari, Matthew Heider, Andrew Rajaram, Sanjeeth Neal, Joel Diehn, Maximilian Zhou, Xiang Li, Ruijiang |
| author_facet | Xiang, Jinxi Hou, Siyu Li, Yuchen Quinton, Ryan Zhang, Xiaoming Eweje, Feyisope Luo, Xiangde Chen, Yijiang Li, Zhe Bergstrom, Colin Kim, Ted Willens, Sierra Olguin, Francesca Maria Abikenari, Matthew Heider, Andrew Rajaram, Sanjeeth Neal, Joel Diehn, Maximilian Zhou, Xiang Li, Ruijiang |
| contents | Spatial transcriptomics (ST) enables gene expression mapping within anatomical context but remains costly and low-throughput. Hematoxylin and eosin (H\&E) staining offers rich morphology yet lacks molecular resolution. We present \textbf{\ours} (\textbf{S}patial \textbf{T}ranscriptomics and hist\textbf{O}logy \textbf{R}epresentation \textbf{M}odel), a foundation model trained on 1.2 million spatially resolved transcriptomic profiles with matched histology across 18 organs. Using a hierarchical architecture integrating morphological features, gene expression, and spatial context, STORM bridges imaging and omics through robust molecular--morphological representations. STORM enhances spatial domain discovery, producing biologically coherent tissue maps, and outperforms existing methods in predicting spatial gene expression from H\&E images across 11 tumor types. The model is platform-agnostic, performing consistently across Visium, Xenium, Visium HD, and CosMx. Applied to 23 independent cohorts comprising 7,245 patients, STORM significantly improves immunotherapy response prediction and prognostication over established biomarkers, providing a scalable framework for spatially informed discovery and clinical precision medicine. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_03630 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction Xiang, Jinxi Hou, Siyu Li, Yuchen Quinton, Ryan Zhang, Xiaoming Eweje, Feyisope Luo, Xiangde Chen, Yijiang Li, Zhe Bergstrom, Colin Kim, Ted Willens, Sierra Olguin, Francesca Maria Abikenari, Matthew Heider, Andrew Rajaram, Sanjeeth Neal, Joel Diehn, Maximilian Zhou, Xiang Li, Ruijiang Artificial Intelligence Quantitative Methods Spatial transcriptomics (ST) enables gene expression mapping within anatomical context but remains costly and low-throughput. Hematoxylin and eosin (H\&E) staining offers rich morphology yet lacks molecular resolution. We present \textbf{\ours} (\textbf{S}patial \textbf{T}ranscriptomics and hist\textbf{O}logy \textbf{R}epresentation \textbf{M}odel), a foundation model trained on 1.2 million spatially resolved transcriptomic profiles with matched histology across 18 organs. Using a hierarchical architecture integrating morphological features, gene expression, and spatial context, STORM bridges imaging and omics through robust molecular--morphological representations. STORM enhances spatial domain discovery, producing biologically coherent tissue maps, and outperforms existing methods in predicting spatial gene expression from H\&E images across 11 tumor types. The model is platform-agnostic, performing consistently across Visium, Xenium, Visium HD, and CosMx. Applied to 23 independent cohorts comprising 7,245 patients, STORM significantly improves immunotherapy response prediction and prognostication over established biomarkers, providing a scalable framework for spatially informed discovery and clinical precision medicine. |
| title | A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction |
| topic | Artificial Intelligence Quantitative Methods |
| url | https://arxiv.org/abs/2604.03630 |