Addressing Reproducibility Challenges in HPC with Continuous Integration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915470106427392 |
|---|---|
| author | Hayot-Sasson, Valérie Hudson, Nathaniel Bauer, André Gonthier, Maxime Foster, Ian Chard, Kyle |
| author_facet | Hayot-Sasson, Valérie Hudson, Nathaniel Bauer, André Gonthier, Maxime Foster, Ian Chard, Kyle |
| contents | The high-performance computing (HPC) community has adopted incentive structures to motivate reproducible research, with major conferences awarding badges to papers that meet reproducibility requirements. Yet, many papers do not meet such requirements. The uniqueness of HPC infrastructure and software, coupled with strict access requirements, may limit opportunities for reproducibility. In the absence of resource access, we believe that regular documented testing, through continuous integration (CI), coupled with complete provenance information, can be used as a substitute. Here, we argue that better HPC-compliant CI solutions will improve reproducibility of applications. We present a survey of reproducibility initiatives and describe the barriers to reproducibility in HPC. To address existing limitations, we present a GitHub Action, CORRECT, that enables secure execution of tests on remote HPC resources. We evaluate CORRECT's usability across three different types of HPC applications, demonstrating the effectiveness of using CORRECT for automating and documenting reproducibility evaluations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_21289 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Addressing Reproducibility Challenges in HPC with Continuous Integration Hayot-Sasson, Valérie Hudson, Nathaniel Bauer, André Gonthier, Maxime Foster, Ian Chard, Kyle Distributed, Parallel, and Cluster Computing Software Engineering The high-performance computing (HPC) community has adopted incentive structures to motivate reproducible research, with major conferences awarding badges to papers that meet reproducibility requirements. Yet, many papers do not meet such requirements. The uniqueness of HPC infrastructure and software, coupled with strict access requirements, may limit opportunities for reproducibility. In the absence of resource access, we believe that regular documented testing, through continuous integration (CI), coupled with complete provenance information, can be used as a substitute. Here, we argue that better HPC-compliant CI solutions will improve reproducibility of applications. We present a survey of reproducibility initiatives and describe the barriers to reproducibility in HPC. To address existing limitations, we present a GitHub Action, CORRECT, that enables secure execution of tests on remote HPC resources. We evaluate CORRECT's usability across three different types of HPC applications, demonstrating the effectiveness of using CORRECT for automating and documenting reproducibility evaluations. |
| title | Addressing Reproducibility Challenges in HPC with Continuous Integration |
| topic | Distributed, Parallel, and Cluster Computing Software Engineering |
| url | https://arxiv.org/abs/2508.21289 |