Scalable HPC Job Scheduling and Resource Management in SST
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909470078009344 |
|---|---|
| author | Abdurahman, Abubeker Hossain, Abrar Brown, Kevin A Yoshii, Kazutomo Ahmed, Kishwar |
| author_facet | Abdurahman, Abubeker Hossain, Abrar Brown, Kevin A Yoshii, Kazutomo Ahmed, Kishwar |
| contents | Efficient job scheduling and resource management contribute towards system throughput and efficiency maximization in high-performance computing (HPC) systems. In this paper, we introduce a scalable job scheduling and resource management component within the structural simulation toolkit (SST), a cycle-accurate and parallel discrete-event simulator. Our proposed simulator includes state-of-the-art job scheduling algorithms and resource management techniques. Additionally, it introduces workflow management components that support the simulation of task dependencies and resource allocations, crucial for workflows typical in scientific computing and data-intensive applications. We present the validation and scalability results of our job scheduling simulator. Simulation shows that our simulator achieves good accuracy in various metrics (e.g., job wait times, number of nodes usage) and also achieves good parallel performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_18191 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scalable HPC Job Scheduling and Resource Management in SST Abdurahman, Abubeker Hossain, Abrar Brown, Kevin A Yoshii, Kazutomo Ahmed, Kishwar Distributed, Parallel, and Cluster Computing Efficient job scheduling and resource management contribute towards system throughput and efficiency maximization in high-performance computing (HPC) systems. In this paper, we introduce a scalable job scheduling and resource management component within the structural simulation toolkit (SST), a cycle-accurate and parallel discrete-event simulator. Our proposed simulator includes state-of-the-art job scheduling algorithms and resource management techniques. Additionally, it introduces workflow management components that support the simulation of task dependencies and resource allocations, crucial for workflows typical in scientific computing and data-intensive applications. We present the validation and scalability results of our job scheduling simulator. Simulation shows that our simulator achieves good accuracy in various metrics (e.g., job wait times, number of nodes usage) and also achieves good parallel performance. |
| title | Scalable HPC Job Scheduling and Resource Management in SST |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2501.18191 |