A Communication- and Memory-Aware Model for Load Balancing Tasks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917650506973184 |
|---|---|
| author | Lifflander, Jonathan Pebay, Philippe P. Slattengren, Nicole L. Pebay, Pierre L. Pfeiffer, Robert A. Kotulski, Joseph D. McGovern, Sean T. |
| author_facet | Lifflander, Jonathan Pebay, Philippe P. Slattengren, Nicole L. Pebay, Pierre L. Pfeiffer, Robert A. Kotulski, Joseph D. McGovern, Sean T. |
| contents | While load balancing in distributed-memory computing has been well-studied, we present an innovative approach to this problem: a unified, reduced-order model that combines three key components to describe "work" in a distributed system: computation, communication, and memory. Our model enables an optimizer to explore complex tradeoffs in task placement, such as increased parallelism at the expense of data replication, which increases memory usage. We propose a fully distributed, heuristic-based load balancing optimization algorithm, and demonstrate that it quickly finds close-to-optimal solutions. We formalize the complex optimization problem as a mixed-integer linear program, and compare it to our strategy. Finally, we show that when applied to an electromagnetics code, our approach obtains up to 2.3x speedups for the imbalanced execution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_16793 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Communication- and Memory-Aware Model for Load Balancing Tasks Lifflander, Jonathan Pebay, Philippe P. Slattengren, Nicole L. Pebay, Pierre L. Pfeiffer, Robert A. Kotulski, Joseph D. McGovern, Sean T. Distributed, Parallel, and Cluster Computing While load balancing in distributed-memory computing has been well-studied, we present an innovative approach to this problem: a unified, reduced-order model that combines three key components to describe "work" in a distributed system: computation, communication, and memory. Our model enables an optimizer to explore complex tradeoffs in task placement, such as increased parallelism at the expense of data replication, which increases memory usage. We propose a fully distributed, heuristic-based load balancing optimization algorithm, and demonstrate that it quickly finds close-to-optimal solutions. We formalize the complex optimization problem as a mixed-integer linear program, and compare it to our strategy. Finally, we show that when applied to an electromagnetics code, our approach obtains up to 2.3x speedups for the imbalanced execution. |
| title | A Communication- and Memory-Aware Model for Load Balancing Tasks |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2404.16793 |