Online Learning of Weakly Coupled MDP Policies for Load Balancing and Auto Scaling
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911927343513600 |
|---|---|
| author | Eshwar, S. R. Felipe, Lucas Lopes Reiffers-Masson, Alexandre Menasché, Daniel Sadoc Thoppe, Gugan |
| author_facet | Eshwar, S. R. Felipe, Lucas Lopes Reiffers-Masson, Alexandre Menasché, Daniel Sadoc Thoppe, Gugan |
| contents | Load balancing and auto scaling are at the core of scalable, contemporary systems, addressing dynamic resource allocation and service rate adjustments in response to workload changes. This paper introduces a novel model and algorithms for tuning load balancers coupled with auto scalers, considering bursty traffic arriving at finite queues. We begin by presenting the problem as a weakly coupled Markov Decision Processes (MDP), solvable via a linear program (LP). However, as the number of control variables of such LP grows combinatorially, we introduce a more tractable relaxed LP formulation, and extend it to tackle the problem of online parameter learning and policy optimization using a two-timescale algorithm based on the LP Lagrangian. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_14141 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Online Learning of Weakly Coupled MDP Policies for Load Balancing and Auto Scaling Eshwar, S. R. Felipe, Lucas Lopes Reiffers-Masson, Alexandre Menasché, Daniel Sadoc Thoppe, Gugan Systems and Control Artificial Intelligence Networking and Internet Architecture Load balancing and auto scaling are at the core of scalable, contemporary systems, addressing dynamic resource allocation and service rate adjustments in response to workload changes. This paper introduces a novel model and algorithms for tuning load balancers coupled with auto scalers, considering bursty traffic arriving at finite queues. We begin by presenting the problem as a weakly coupled Markov Decision Processes (MDP), solvable via a linear program (LP). However, as the number of control variables of such LP grows combinatorially, we introduce a more tractable relaxed LP formulation, and extend it to tackle the problem of online parameter learning and policy optimization using a two-timescale algorithm based on the LP Lagrangian. |
| title | Online Learning of Weakly Coupled MDP Policies for Load Balancing and Auto Scaling |
| topic | Systems and Control Artificial Intelligence Networking and Internet Architecture |
| url | https://arxiv.org/abs/2406.14141 |