Enhancing Cluster Scheduling in HPC: A Continuous Transfer Learning for Real-Time Optimization
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916972849004544 |
|---|---|
| author | Sliwko, Leszek Mizera-Pietraszko, Jolanta |
| author_facet | Sliwko, Leszek Mizera-Pietraszko, Jolanta |
| contents | This study presents a machine learning-assisted approach to optimize task scheduling in cluster systems, focusing on node-affinity constraints. Traditional schedulers like Kubernetes struggle with real-time adaptability, whereas the proposed continuous transfer learning model evolves dynamically during operations, minimizing retraining needs. Evaluated on Google Cluster Data, the model achieves over 99% accuracy, reducing computational overhead and improving scheduling latency for constrained tasks. This scalable solution enables real-time optimization, advancing machine learning integration in cluster management and paving the way for future adaptive scheduling strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_22701 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Enhancing Cluster Scheduling in HPC: A Continuous Transfer Learning for Real-Time Optimization Sliwko, Leszek Mizera-Pietraszko, Jolanta Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning This study presents a machine learning-assisted approach to optimize task scheduling in cluster systems, focusing on node-affinity constraints. Traditional schedulers like Kubernetes struggle with real-time adaptability, whereas the proposed continuous transfer learning model evolves dynamically during operations, minimizing retraining needs. Evaluated on Google Cluster Data, the model achieves over 99% accuracy, reducing computational overhead and improving scheduling latency for constrained tasks. This scalable solution enables real-time optimization, advancing machine learning integration in cluster management and paving the way for future adaptive scheduling strategies. |
| title | Enhancing Cluster Scheduling in HPC: A Continuous Transfer Learning for Real-Time Optimization |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2509.22701 |