Capacity Planning and Scheduling for Jobs with Uncertainty in Resource Usage and Duration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912495919169536 |
|---|---|
| author | Patra, Sunandita Pathan, Mehtab Mahfouz, Mahmoud Zehtabi, Parisa Ouaja, Wided Magazzeni, Daniele Veloso, Manuela |
| author_facet | Patra, Sunandita Pathan, Mehtab Mahfouz, Mahmoud Zehtabi, Parisa Ouaja, Wided Magazzeni, Daniele Veloso, Manuela |
| contents | Organizations around the world schedule jobs (programs) regularly to perform various tasks dictated by their end users. With the major movement towards using a cloud computing infrastructure, our organization follows a hybrid approach with both cloud and on-prem servers. The objective of this work is to perform capacity planning, i.e., estimate resource requirements, and job scheduling for on-prem grid computing environments. A key contribution of our approach is handling uncertainty in both resource usage and duration of the jobs, a critical aspect in the finance industry where stochastic market conditions significantly influence job characteristics. For capacity planning and scheduling, we simultaneously balance two conflicting objectives: (a) minimize resource usage, and (b) provide high quality-of-service to the end users by completing jobs by their requested deadlines. We propose approximate approaches using deterministic estimators and pair sampling-based constraint programming. Our best approach (pair sampling-based) achieves much lower peak resource usage compared to manual scheduling without compromising on the quality-of-service. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_01225 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Capacity Planning and Scheduling for Jobs with Uncertainty in Resource Usage and Duration Patra, Sunandita Pathan, Mehtab Mahfouz, Mahmoud Zehtabi, Parisa Ouaja, Wided Magazzeni, Daniele Veloso, Manuela Distributed, Parallel, and Cluster Computing Artificial Intelligence Organizations around the world schedule jobs (programs) regularly to perform various tasks dictated by their end users. With the major movement towards using a cloud computing infrastructure, our organization follows a hybrid approach with both cloud and on-prem servers. The objective of this work is to perform capacity planning, i.e., estimate resource requirements, and job scheduling for on-prem grid computing environments. A key contribution of our approach is handling uncertainty in both resource usage and duration of the jobs, a critical aspect in the finance industry where stochastic market conditions significantly influence job characteristics. For capacity planning and scheduling, we simultaneously balance two conflicting objectives: (a) minimize resource usage, and (b) provide high quality-of-service to the end users by completing jobs by their requested deadlines. We propose approximate approaches using deterministic estimators and pair sampling-based constraint programming. Our best approach (pair sampling-based) achieves much lower peak resource usage compared to manual scheduling without compromising on the quality-of-service. |
| title | Capacity Planning and Scheduling for Jobs with Uncertainty in Resource Usage and Duration |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence |
| url | https://arxiv.org/abs/2507.01225 |