Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.05614 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912695669751808 |
|---|---|
| author | Hawks, Ben von Laszewski, Gregor Sinclair, Matthew D. Colombo, Marco Venkataraman, Shivaram Jain, Rutwik Jiang, Yiwei Tran, Nhan Fox, Geoffrey |
| author_facet | Hawks, Ben von Laszewski, Gregor Sinclair, Matthew D. Colombo, Marco Venkataraman, Shivaram Jain, Rutwik Jiang, Yiwei Tran, Nhan Fox, Geoffrey |
| contents | Scientific machine learning research spans diverse domains and data modalities, yet existing benchmark efforts remain siloed and lack standardization. This makes novel and transformative applications of machine learning to critical scientific use-cases more fragmented and less clear in pathways to impact. This paper introduces an ontology for scientific benchmarking developed through a unified, community-driven effort that extends the MLCommons ecosystem to cover physics, chemistry, materials science, biology, climate science, and more. Building on prior initiatives such as XAI-BENCH, FastML Science Benchmarks, PDEBench, and the SciMLBench framework, our effort consolidates a large set of disparate benchmarks and frameworks into a single taxonomy of scientific, application, and system-level benchmarks. New benchmarks can be added through an open submission workflow coordinated by the MLCommons Science Working Group and evaluated against a six-category rating rubric that promotes and identifies high-quality benchmarks, enabling stakeholders to select benchmarks that meet their specific needs. The architecture is extensible, supporting future scientific and AI/ML motifs, and we discuss methods for identifying emerging computing patterns for unique scientific workloads. The MLCommons Science Benchmarks Ontology provides a standardized, scalable foundation for reproducible, cross-domain benchmarking in scientific machine learning. A companion webpage for this work has also been developed as the effort evolves: https://mlcommons-science.github.io/benchmark/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_05614 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An MLCommons Scientific Benchmarks Ontology Hawks, Ben von Laszewski, Gregor Sinclair, Matthew D. Colombo, Marco Venkataraman, Shivaram Jain, Rutwik Jiang, Yiwei Tran, Nhan Fox, Geoffrey Machine Learning Artificial Intelligence Performance Computational Physics Scientific machine learning research spans diverse domains and data modalities, yet existing benchmark efforts remain siloed and lack standardization. This makes novel and transformative applications of machine learning to critical scientific use-cases more fragmented and less clear in pathways to impact. This paper introduces an ontology for scientific benchmarking developed through a unified, community-driven effort that extends the MLCommons ecosystem to cover physics, chemistry, materials science, biology, climate science, and more. Building on prior initiatives such as XAI-BENCH, FastML Science Benchmarks, PDEBench, and the SciMLBench framework, our effort consolidates a large set of disparate benchmarks and frameworks into a single taxonomy of scientific, application, and system-level benchmarks. New benchmarks can be added through an open submission workflow coordinated by the MLCommons Science Working Group and evaluated against a six-category rating rubric that promotes and identifies high-quality benchmarks, enabling stakeholders to select benchmarks that meet their specific needs. The architecture is extensible, supporting future scientific and AI/ML motifs, and we discuss methods for identifying emerging computing patterns for unique scientific workloads. The MLCommons Science Benchmarks Ontology provides a standardized, scalable foundation for reproducible, cross-domain benchmarking in scientific machine learning. A companion webpage for this work has also been developed as the effort evolves: https://mlcommons-science.github.io/benchmark/ |
| title | An MLCommons Scientific Benchmarks Ontology |
| topic | Machine Learning Artificial Intelligence Performance Computational Physics |
| url | https://arxiv.org/abs/2511.05614 |