Megha: Decentralized Global Fair Scheduling for Federated Clusters
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2021
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916143967502336 |
|---|---|
| author | Thiyyakat, Meghana Kalambur, Subramaniam Sitaram, Dinkar |
| author_facet | Thiyyakat, Meghana Kalambur, Subramaniam Sitaram, Dinkar |
| contents | Increasing scale and heterogeneity in data centers have led to the development of federated clusters such as KubeFed, Hydra, and Pigeon, that federate individual data center clusters. In our work, we introduce Megha, a novel decentralized resource management framework for such federated clusters. Megha employs flexible logical partitioning of clusters to distribute its scheduling load, ensuring that the requirements of the workload are satisfied with very low scheduling overheads. It uses a distributed global scheduler that does not rely on a centralized data store but, instead, works with eventual consistency, unlike other schedulers that use a tiered architecture or rely on centralized databases. Our experiments with Megha show that it can schedule tasks taking into account fairness and placement constraints with low resource allocation times - in the order of tens of milliseconds. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2103_08413 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | Megha: Decentralized Global Fair Scheduling for Federated Clusters Thiyyakat, Meghana Kalambur, Subramaniam Sitaram, Dinkar Distributed, Parallel, and Cluster Computing Increasing scale and heterogeneity in data centers have led to the development of federated clusters such as KubeFed, Hydra, and Pigeon, that federate individual data center clusters. In our work, we introduce Megha, a novel decentralized resource management framework for such federated clusters. Megha employs flexible logical partitioning of clusters to distribute its scheduling load, ensuring that the requirements of the workload are satisfied with very low scheduling overheads. It uses a distributed global scheduler that does not rely on a centralized data store but, instead, works with eventual consistency, unlike other schedulers that use a tiered architecture or rely on centralized databases. Our experiments with Megha show that it can schedule tasks taking into account fairness and placement constraints with low resource allocation times - in the order of tens of milliseconds. |
| title | Megha: Decentralized Global Fair Scheduling for Federated Clusters |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2103.08413 |