PanFoMa: A Lightweight Foundation Model and Benchmark for Pan-Cancer
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909941333229568 |
|---|---|
| author | Huang, Xiaoshui Zhu, Tianlin Zuo, Yifan Xia, Xue Wu, Zonghan Yan, Jiebin Hua, Dingli Xu, Zongyi Fang, Yuming Zhang, Jian |
| author_facet | Huang, Xiaoshui Zhu, Tianlin Zuo, Yifan Xia, Xue Wu, Zonghan Yan, Jiebin Hua, Dingli Xu, Zongyi Fang, Yuming Zhang, Jian |
| contents | Single-cell RNA sequencing (scRNA-seq) is essential for decoding tumor heterogeneity. However, pan-cancer research still faces two key challenges: learning discriminative and efficient single-cell representations, and establishing a comprehensive evaluation benchmark. In this paper, we introduce PanFoMa, a lightweight hybrid neural network that combines the strengths of Transformers and state-space models to achieve a balance between performance and efficiency. PanFoMa consists of a front-end local-context encoder with shared self-attention layers to capture complex, order-independent gene interactions; and a back-end global sequential feature decoder that efficiently integrates global context using a linear-time state-space model. This modular design preserves the expressive power of Transformers while leveraging the scalability of Mamba to enable transcriptome modeling, effectively capturing both local and global regulatory signals. To enable robust evaluation, we also construct a large-scale pan-cancer single-cell benchmark, PanFoMaBench, containing over 3.5 million high-quality cells across 33 cancer subtypes, curated through a rigorous preprocessing pipeline. Experimental results show that PanFoMa outperforms state-of-the-art models on our pan-cancer benchmark (+4.0\%) and across multiple public tasks, including cell type annotation (+7.4\%), batch integration (+4.0\%) and multi-omics integration (+3.1\%). The code is available at https://github.com/Xiaoshui-Huang/PanFoMa. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_03111 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PanFoMa: A Lightweight Foundation Model and Benchmark for Pan-Cancer Huang, Xiaoshui Zhu, Tianlin Zuo, Yifan Xia, Xue Wu, Zonghan Yan, Jiebin Hua, Dingli Xu, Zongyi Fang, Yuming Zhang, Jian Genomics Artificial Intelligence Computer Vision and Pattern Recognition Single-cell RNA sequencing (scRNA-seq) is essential for decoding tumor heterogeneity. However, pan-cancer research still faces two key challenges: learning discriminative and efficient single-cell representations, and establishing a comprehensive evaluation benchmark. In this paper, we introduce PanFoMa, a lightweight hybrid neural network that combines the strengths of Transformers and state-space models to achieve a balance between performance and efficiency. PanFoMa consists of a front-end local-context encoder with shared self-attention layers to capture complex, order-independent gene interactions; and a back-end global sequential feature decoder that efficiently integrates global context using a linear-time state-space model. This modular design preserves the expressive power of Transformers while leveraging the scalability of Mamba to enable transcriptome modeling, effectively capturing both local and global regulatory signals. To enable robust evaluation, we also construct a large-scale pan-cancer single-cell benchmark, PanFoMaBench, containing over 3.5 million high-quality cells across 33 cancer subtypes, curated through a rigorous preprocessing pipeline. Experimental results show that PanFoMa outperforms state-of-the-art models on our pan-cancer benchmark (+4.0\%) and across multiple public tasks, including cell type annotation (+7.4\%), batch integration (+4.0\%) and multi-omics integration (+3.1\%). The code is available at https://github.com/Xiaoshui-Huang/PanFoMa. |
| title | PanFoMa: A Lightweight Foundation Model and Benchmark for Pan-Cancer |
| topic | Genomics Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.03111 |