Parallel I/O Characterization and Optimization on Large-Scale HPC Systems: A 360-Degree Survey
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917882017873920 |
|---|---|
| author | Ather, Hammad Bez, Jean Luca Wang, Chen Childs, Hank Malony, Allen D. Byna, Suren |
| author_facet | Ather, Hammad Bez, Jean Luca Wang, Chen Childs, Hank Malony, Allen D. Byna, Suren |
| contents | Driven by artificial intelligence, data science, and high-resolution simulations, I/O workloads and hardware on high-performance computing (HPC) systems have become increasingly complex. This complexity can lead to large I/O overheads and overall performance degradation. These inefficiencies are often mitigated using tools and techniques for characterizing, analyzing, and optimizing the I/O behavior of HPC applications. That said, the myriad number of tools and techniques available makes it challenging to navigate to the best approach. In response, this paper surveys 131 papers from the ACM Digital Library, IEEE Xplore, and other reputable journals to provide a comprehensive analysis, synthesized in the form of a taxonomy, of the current landscape of parallel I/O characterization, analysis, and optimization of large-scale HPC systems. We anticipate that this taxonomy will serve as a valuable resource for enhancing I/O performance of HPC applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_00203 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Parallel I/O Characterization and Optimization on Large-Scale HPC Systems: A 360-Degree Survey Ather, Hammad Bez, Jean Luca Wang, Chen Childs, Hank Malony, Allen D. Byna, Suren Distributed, Parallel, and Cluster Computing Performance Driven by artificial intelligence, data science, and high-resolution simulations, I/O workloads and hardware on high-performance computing (HPC) systems have become increasingly complex. This complexity can lead to large I/O overheads and overall performance degradation. These inefficiencies are often mitigated using tools and techniques for characterizing, analyzing, and optimizing the I/O behavior of HPC applications. That said, the myriad number of tools and techniques available makes it challenging to navigate to the best approach. In response, this paper surveys 131 papers from the ACM Digital Library, IEEE Xplore, and other reputable journals to provide a comprehensive analysis, synthesized in the form of a taxonomy, of the current landscape of parallel I/O characterization, analysis, and optimization of large-scale HPC systems. We anticipate that this taxonomy will serve as a valuable resource for enhancing I/O performance of HPC applications. |
| title | Parallel I/O Characterization and Optimization on Large-Scale HPC Systems: A 360-Degree Survey |
| topic | Distributed, Parallel, and Cluster Computing Performance |
| url | https://arxiv.org/abs/2501.00203 |