Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.07938 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915994042105856 |
|---|---|
| author | Weerasekara, Sachini Darras, Natasha Kamarthi, Sagar Price, Colles Isaacs, Jacqueline |
| author_facet | Weerasekara, Sachini Darras, Natasha Kamarthi, Sagar Price, Colles Isaacs, Jacqueline |
| contents | Single-cell representation learning (SCRL) from gene expression data offers a way to uncover the complex regulatory logic underlying cellular function. Inspired by large language models in natural language modeling, several single-cell pretrained models have recently been proposed that treat genes as tokens and cells as sentences. However, these models are fundamentally limited by the long-tailed nature of cell-type distributions and struggle to generalize under covariate shifts in gene expression data. While fine-tuning is often used to mitigate these issues, we observe that performance remains bounded. To address this challenge, we introduce CellRefine, a post-pretraining method that operates between the pretraining and fine-tuning stages of a single-cell foundation model. CellRefine uses a multi-faceted objective that incorporates marker-gene sets as structural priors to guide post-pretraining and refine the latent embedding manifold of cells. Across multiple computational biology tasks, empirical results show that CellRefine consistently improves downstream performance, yielding gains up to 15%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_07938 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Prototype Guided Post-pretraining for Single-Cell Representation Learning Weerasekara, Sachini Darras, Natasha Kamarthi, Sagar Price, Colles Isaacs, Jacqueline Machine Learning Single-cell representation learning (SCRL) from gene expression data offers a way to uncover the complex regulatory logic underlying cellular function. Inspired by large language models in natural language modeling, several single-cell pretrained models have recently been proposed that treat genes as tokens and cells as sentences. However, these models are fundamentally limited by the long-tailed nature of cell-type distributions and struggle to generalize under covariate shifts in gene expression data. While fine-tuning is often used to mitigate these issues, we observe that performance remains bounded. To address this challenge, we introduce CellRefine, a post-pretraining method that operates between the pretraining and fine-tuning stages of a single-cell foundation model. CellRefine uses a multi-faceted objective that incorporates marker-gene sets as structural priors to guide post-pretraining and refine the latent embedding manifold of cells. Across multiple computational biology tasks, empirical results show that CellRefine consistently improves downstream performance, yielding gains up to 15%. |
| title | Prototype Guided Post-pretraining for Single-Cell Representation Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2605.07938 |