Towards Experiment Execution in Support of Community Benchmark Workflows for HPC
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912510768054272 |
|---|---|
| author | von Laszewski, Gregor Brewer, Wesley Wilkinson, Sean R. Shao, Andrew Fleischer, J. P. Pitkar, Harshad Kirkpatrick, Christine R. Fox, Geoffrey C. |
| author_facet | von Laszewski, Gregor Brewer, Wesley Wilkinson, Sean R. Shao, Andrew Fleischer, J. P. Pitkar, Harshad Kirkpatrick, Christine R. Fox, Geoffrey C. |
| contents | A key hurdle is demonstrating compute resource capability with limited benchmarks. We propose workflow templates as a solution, offering adaptable designs for specific scientific applications. Our paper identifies common usage patterns for these templates, drawn from decades of HPC experience, including recent work with the MLCommons Science working group.
We found that focusing on simple experiment management tools within the broader computational workflow improves adaptability, especially in education. This concept, which we term benchmark carpentry, is validated by two independent tools: Cloudmesh's Experiment Executor and Hewlett Packard Enterprise's SmartSim. Both frameworks, with significant functional overlap, have been tested across various scientific applications, including conduction cloudmask, earthquake prediction, simulation-AI/ML interactions, and the development of computational fluid dynamics surrogates. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_22294 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Experiment Execution in Support of Community Benchmark Workflows for HPC von Laszewski, Gregor Brewer, Wesley Wilkinson, Sean R. Shao, Andrew Fleischer, J. P. Pitkar, Harshad Kirkpatrick, Christine R. Fox, Geoffrey C. Distributed, Parallel, and Cluster Computing A key hurdle is demonstrating compute resource capability with limited benchmarks. We propose workflow templates as a solution, offering adaptable designs for specific scientific applications. Our paper identifies common usage patterns for these templates, drawn from decades of HPC experience, including recent work with the MLCommons Science working group. We found that focusing on simple experiment management tools within the broader computational workflow improves adaptability, especially in education. This concept, which we term benchmark carpentry, is validated by two independent tools: Cloudmesh's Experiment Executor and Hewlett Packard Enterprise's SmartSim. Both frameworks, with significant functional overlap, have been tested across various scientific applications, including conduction cloudmask, earthquake prediction, simulation-AI/ML interactions, and the development of computational fluid dynamics surrogates. |
| title | Towards Experiment Execution in Support of Community Benchmark Workflows for HPC |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2507.22294 |