Diversity by Design: Addressing Mode Collapse Improves scRNA-seq Perturbation Modeling on Well-Calibrated Metrics

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Mejia, Gabriel M., Miller, Henry E., Leblanc, Francis J. A., Wang, Bo, Swain, Brendan, Camillo, Lucas Paulo de Lima
Format:	Preprint
Veröffentlicht:	2025
Schlagworte:	Genomics Machine Learning Molecular Networks
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

_version_	1866918074605633536
author	Mejia, Gabriel M. Miller, Henry E. Leblanc, Francis J. A. Wang, Bo Swain, Brendan Camillo, Lucas Paulo de Lima
author_facet	Mejia, Gabriel M. Miller, Henry E. Leblanc, Francis J. A. Wang, Bo Swain, Brendan Camillo, Lucas Paulo de Lima
contents	Recent benchmarks reveal that models for single-cell perturbation response are often outperformed by simply predicting the dataset mean. We trace this anomaly to a metric artifact: control-referenced deltas and unweighted error metrics reward mode collapse whenever the control is biased or the biological signal is sparse. Large-scale \textit{in silico} simulations and analysis of two real-world perturbation datasets confirm that shared reference shifts, not genuine biological change, drives high performance in these evaluations. We introduce differentially expressed gene (DEG)-aware metrics, weighted mean-squared error (WMSE) and weighted delta $R^{2}$ ($R^{2}_{w}(Δ)$) with respect to all perturbations, that measure error in niche signals with high sensitivity. We further introduce negative and positive performance baselines to calibrate these metrics. With these improvements, the mean baseline sinks to null performance while genuine predictors are correctly rewarded. Finally, we show that using WMSE as a loss function reduces mode collapse and improves model performance.
format	Preprint
id	arxiv_https___arxiv_org_abs_2506_22641
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Diversity by Design: Addressing Mode Collapse Improves scRNA-seq Perturbation Modeling on Well-Calibrated Metrics Mejia, Gabriel M. Miller, Henry E. Leblanc, Francis J. A. Wang, Bo Swain, Brendan Camillo, Lucas Paulo de Lima Genomics Machine Learning Molecular Networks Recent benchmarks reveal that models for single-cell perturbation response are often outperformed by simply predicting the dataset mean. We trace this anomaly to a metric artifact: control-referenced deltas and unweighted error metrics reward mode collapse whenever the control is biased or the biological signal is sparse. Large-scale \textit{in silico} simulations and analysis of two real-world perturbation datasets confirm that shared reference shifts, not genuine biological change, drives high performance in these evaluations. We introduce differentially expressed gene (DEG)-aware metrics, weighted mean-squared error (WMSE) and weighted delta $R^{2}$ ($R^{2}_{w}(Δ)$) with respect to all perturbations, that measure error in niche signals with high sensitivity. We further introduce negative and positive performance baselines to calibrate these metrics. With these improvements, the mean baseline sinks to null performance while genuine predictors are correctly rewarded. Finally, we show that using WMSE as a loss function reduces mode collapse and improves model performance.
title	Diversity by Design: Addressing Mode Collapse Improves scRNA-seq Perturbation Modeling on Well-Calibrated Metrics
topic	Genomics Machine Learning Molecular Networks
url	https://arxiv.org/abs/2506.22641