AssayBench: An Assay-Level Virtual Cell Benchmark for LLMs and Agents
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913112877170688 |
|---|---|
| author | De Brouwer, Edward Edwards, Carl Wu, Alexander Collier, Jenna Heimberg, Graham Li, Xiner Subramaniam, Meena Hajiramezanali, Ehsan Richmond, David Hütter, Jan-Christian Mostafavi, Sara Scalia, Gabriele |
| author_facet | De Brouwer, Edward Edwards, Carl Wu, Alexander Collier, Jenna Heimberg, Graham Li, Xiner Subramaniam, Meena Hajiramezanali, Ehsan Richmond, David Hütter, Jan-Christian Mostafavi, Sara Scalia, Gabriele |
| contents | Recent advances in machine learning and large-scale biological data collections have revived the prospect of building a virtual cell, a computational model of cellular behavior that could accelerate biological discovery. One of the most compelling promises of this vision is the ability to perform in silico phenotypic screens, in which a model predicts the effects of cellular perturbations in unseen biological contexts. This task combines heterogeneous textual inputs with diverse phenotypic outputs, making it particularly well-suited to LLMs and agentic systems. Yet, no standard benchmark currently exists for this task, as existing efforts focus on narrower molecular readouts that are only indirectly aligned with the phenotypic endpoints driving many real-world drug discovery workflows. In this work, we present AssayBench, a benchmark for phenotypic screen prediction, built from 1,920 publicly available CRISPR screens spanning five broad classes of cellular phenotypes. We formulate the screen prediction task as a gene rank prediction for each screen and introduce the adjusted nDCG, a continuous metric for comparing performance across heterogeneous assays. Our extensive evaluation shows that existing methods remain far from empirically estimated performance ceilings and zero-shot generalist LLMs outperform biology-specific LLMs and trainable baselines. Optimization techniques such as fine-tuning, ensembling, and prompt optimization can further improve LLM performance on this task. Overall, AssayBench offers a practical testbed for measuring progress toward in silico phenotypic screening and, more broadly, virtual cell models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_10876 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AssayBench: An Assay-Level Virtual Cell Benchmark for LLMs and Agents De Brouwer, Edward Edwards, Carl Wu, Alexander Collier, Jenna Heimberg, Graham Li, Xiner Subramaniam, Meena Hajiramezanali, Ehsan Richmond, David Hütter, Jan-Christian Mostafavi, Sara Scalia, Gabriele Machine Learning Artificial Intelligence Quantitative Methods Recent advances in machine learning and large-scale biological data collections have revived the prospect of building a virtual cell, a computational model of cellular behavior that could accelerate biological discovery. One of the most compelling promises of this vision is the ability to perform in silico phenotypic screens, in which a model predicts the effects of cellular perturbations in unseen biological contexts. This task combines heterogeneous textual inputs with diverse phenotypic outputs, making it particularly well-suited to LLMs and agentic systems. Yet, no standard benchmark currently exists for this task, as existing efforts focus on narrower molecular readouts that are only indirectly aligned with the phenotypic endpoints driving many real-world drug discovery workflows. In this work, we present AssayBench, a benchmark for phenotypic screen prediction, built from 1,920 publicly available CRISPR screens spanning five broad classes of cellular phenotypes. We formulate the screen prediction task as a gene rank prediction for each screen and introduce the adjusted nDCG, a continuous metric for comparing performance across heterogeneous assays. Our extensive evaluation shows that existing methods remain far from empirically estimated performance ceilings and zero-shot generalist LLMs outperform biology-specific LLMs and trainable baselines. Optimization techniques such as fine-tuning, ensembling, and prompt optimization can further improve LLM performance on this task. Overall, AssayBench offers a practical testbed for measuring progress toward in silico phenotypic screening and, more broadly, virtual cell models. |
| title | AssayBench: An Assay-Level Virtual Cell Benchmark for LLMs and Agents |
| topic | Machine Learning Artificial Intelligence Quantitative Methods |
| url | https://arxiv.org/abs/2605.10876 |