Intermediate Layers Encode Optimal Biological Representations in Single-Cell Foundation Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915940439949312 |
|---|---|
| author | Civale, Vincenzo Yuto Semeraro, Roberto Bagdanov, Andrew David Magi, Alberto |
| author_facet | Civale, Vincenzo Yuto Semeraro, Roberto Bagdanov, Andrew David Magi, Alberto |
| contents | Current single-cell foundation model benchmarks universally extract final layer embeddings, assuming these represent optimal feature spaces. We systematically evaluate layer-wise representations from scFoundation (100M parameters) and Tahoe-X1 (1.3B parameters) across trajectory inference and perturbation response prediction. Our analysis reveals that optimal layers are task-dependent (trajectory peaks at 60% depth, 31% above final layers) and context-dependent (perturbation optima shift 0-96% across T cell activation states). Notably, first-layer embeddings outperform all deeper layers in quiescent cells, challenging assumptions about hierarchical feature abstraction. These findings demonstrate that "where" to extract features matters as much as "what" the model learns, necessitating systematic layer evaluation tailored to biological task and cellular context rather than defaulting to final-layer embeddings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_14838 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Intermediate Layers Encode Optimal Biological Representations in Single-Cell Foundation Models Civale, Vincenzo Yuto Semeraro, Roberto Bagdanov, Andrew David Magi, Alberto Artificial Intelligence 92B20, 68T07 J.3 Current single-cell foundation model benchmarks universally extract final layer embeddings, assuming these represent optimal feature spaces. We systematically evaluate layer-wise representations from scFoundation (100M parameters) and Tahoe-X1 (1.3B parameters) across trajectory inference and perturbation response prediction. Our analysis reveals that optimal layers are task-dependent (trajectory peaks at 60% depth, 31% above final layers) and context-dependent (perturbation optima shift 0-96% across T cell activation states). Notably, first-layer embeddings outperform all deeper layers in quiescent cells, challenging assumptions about hierarchical feature abstraction. These findings demonstrate that "where" to extract features matters as much as "what" the model learns, necessitating systematic layer evaluation tailored to biological task and cellular context rather than defaulting to final-layer embeddings. |
| title | Intermediate Layers Encode Optimal Biological Representations in Single-Cell Foundation Models |
| topic | Artificial Intelligence 92B20, 68T07 J.3 |
| url | https://arxiv.org/abs/2604.14838 |