Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.10502 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909690310426624 |
|---|---|
| author | Fahsbender, Elizabeth Andersson, Alma Ash, Jeremy Binder, Polina Burkhardt, Daniel Chang, Benjamin Gerber, Georg K. Gitter, Anthony Godau, Patrick Gupta, Ankit Haliburton, Genevieve He, Siyu Ideker, Trey Jelic, Ivana Khan, Aly Kim, Yang-Joon Krishnapriyan, Aditi Laurent, Jon M. Liu, Tianyu Lundberg, Emma Mehta, Shalin B. Moccia, Rob Pisco, Angela Oliveira Pollard, Katherine S. Ramani, Suresh Saez-Rodriguez, Julio Senbabaoglu, Yasin Simon, Elana Sivanandan, Srinivasan Stolovitzky, Gustavo Valer, Marc Wang, Bo Zhang, Xikun Zou, James Kalantar, Katrina |
| author_facet | Fahsbender, Elizabeth Andersson, Alma Ash, Jeremy Binder, Polina Burkhardt, Daniel Chang, Benjamin Gerber, Georg K. Gitter, Anthony Godau, Patrick Gupta, Ankit Haliburton, Genevieve He, Siyu Ideker, Trey Jelic, Ivana Khan, Aly Kim, Yang-Joon Krishnapriyan, Aditi Laurent, Jon M. Liu, Tianyu Lundberg, Emma Mehta, Shalin B. Moccia, Rob Pisco, Angela Oliveira Pollard, Katherine S. Ramani, Suresh Saez-Rodriguez, Julio Senbabaoglu, Yasin Simon, Elana Sivanandan, Srinivasan Stolovitzky, Gustavo Valer, Marc Wang, Bo Zhang, Xikun Zou, James Kalantar, Katrina |
| contents | Artificial intelligence holds immense promise for transforming biology, yet a lack of standardized, cross domain, benchmarks undermines our ability to build robust, trustworthy models. Here, we present insights from a recent workshop that convened machine learning and computational biology experts across imaging, transcriptomics, proteomics, and genomics to tackle this gap. We identify major technical and systemic bottlenecks such as data heterogeneity and noise, reproducibility challenges, biases, and the fragmented ecosystem of publicly available resources and propose a set of recommendations for building benchmarking frameworks that can efficiently compare ML models of biological systems across tasks and data modalities. By promoting high quality data curation, standardized tooling, comprehensive evaluation metrics, and open, collaborative platforms, we aim to accelerate the development of robust benchmarks for AI driven Virtual Cells. These benchmarks are crucial for ensuring rigor, reproducibility, and biological relevance, and will ultimately advance the field toward integrated models that drive new discoveries, therapeutic insights, and a deeper understanding of cellular systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_10502 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Benchmarking and Evaluation of AI Models in Biology: Outcomes and Recommendations from the CZI Virtual Cells Workshop Fahsbender, Elizabeth Andersson, Alma Ash, Jeremy Binder, Polina Burkhardt, Daniel Chang, Benjamin Gerber, Georg K. Gitter, Anthony Godau, Patrick Gupta, Ankit Haliburton, Genevieve He, Siyu Ideker, Trey Jelic, Ivana Khan, Aly Kim, Yang-Joon Krishnapriyan, Aditi Laurent, Jon M. Liu, Tianyu Lundberg, Emma Mehta, Shalin B. Moccia, Rob Pisco, Angela Oliveira Pollard, Katherine S. Ramani, Suresh Saez-Rodriguez, Julio Senbabaoglu, Yasin Simon, Elana Sivanandan, Srinivasan Stolovitzky, Gustavo Valer, Marc Wang, Bo Zhang, Xikun Zou, James Kalantar, Katrina Machine Learning Artificial Intelligence Artificial intelligence holds immense promise for transforming biology, yet a lack of standardized, cross domain, benchmarks undermines our ability to build robust, trustworthy models. Here, we present insights from a recent workshop that convened machine learning and computational biology experts across imaging, transcriptomics, proteomics, and genomics to tackle this gap. We identify major technical and systemic bottlenecks such as data heterogeneity and noise, reproducibility challenges, biases, and the fragmented ecosystem of publicly available resources and propose a set of recommendations for building benchmarking frameworks that can efficiently compare ML models of biological systems across tasks and data modalities. By promoting high quality data curation, standardized tooling, comprehensive evaluation metrics, and open, collaborative platforms, we aim to accelerate the development of robust benchmarks for AI driven Virtual Cells. These benchmarks are crucial for ensuring rigor, reproducibility, and biological relevance, and will ultimately advance the field toward integrated models that drive new discoveries, therapeutic insights, and a deeper understanding of cellular systems. |
| title | Benchmarking and Evaluation of AI Models in Biology: Outcomes and Recommendations from the CZI Virtual Cells Workshop |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2507.10502 |