Cooperative Multi-Agent Constrained Stochastic Linear Bandits
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910662663340032 |
|---|---|
| author | Afsharrad, Amirhossein Oftadeh, Parisa Moradipari, Ahmadreza Lall, Sanjay |
| author_facet | Afsharrad, Amirhossein Oftadeh, Parisa Moradipari, Ahmadreza Lall, Sanjay |
| contents | In this study, we explore a collaborative multi-agent stochastic linear bandit setting involving a network of $N$ agents that communicate locally to minimize their collective regret while keeping their expected cost under a specified threshold $τ$. Each agent encounters a distinct linear bandit problem characterized by its own reward and cost parameters, i.e., local parameters. The goal of the agents is to determine the best overall action corresponding to the average of these parameters, or so-called global parameters. In each round, an agent is randomly chosen to select an action based on its current knowledge of the system. This chosen action is then executed by all agents, then they observe their individual rewards and costs. We propose a safe distributed upper confidence bound algorithm, so called \textit{MA-OPLB}, and establish a high probability bound on its $T$-round regret. MA-OPLB utilizes an accelerated consensus method, where agents can compute an estimate of the average rewards and costs across the network by communicating the proper information with their neighbors. We show that our regret bound is of order $ \mathcal{O}\left(\frac{d}{τ-c_0}\frac{\log(NT)^2}{\sqrt{N}}\sqrt{\frac{T}{\log(1/|λ_2|)}}\right)$, where $λ_2$ is the second largest (in absolute value) eigenvalue of the communication matrix, and $τ-c_0$ is the known cost gap of a feasible action. We also experimentally show the performance of our proposed algorithm in different network structures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_17382 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Cooperative Multi-Agent Constrained Stochastic Linear Bandits Afsharrad, Amirhossein Oftadeh, Parisa Moradipari, Ahmadreza Lall, Sanjay Machine Learning Multiagent Systems In this study, we explore a collaborative multi-agent stochastic linear bandit setting involving a network of $N$ agents that communicate locally to minimize their collective regret while keeping their expected cost under a specified threshold $τ$. Each agent encounters a distinct linear bandit problem characterized by its own reward and cost parameters, i.e., local parameters. The goal of the agents is to determine the best overall action corresponding to the average of these parameters, or so-called global parameters. In each round, an agent is randomly chosen to select an action based on its current knowledge of the system. This chosen action is then executed by all agents, then they observe their individual rewards and costs. We propose a safe distributed upper confidence bound algorithm, so called \textit{MA-OPLB}, and establish a high probability bound on its $T$-round regret. MA-OPLB utilizes an accelerated consensus method, where agents can compute an estimate of the average rewards and costs across the network by communicating the proper information with their neighbors. We show that our regret bound is of order $ \mathcal{O}\left(\frac{d}{τ-c_0}\frac{\log(NT)^2}{\sqrt{N}}\sqrt{\frac{T}{\log(1/|λ_2|)}}\right)$, where $λ_2$ is the second largest (in absolute value) eigenvalue of the communication matrix, and $τ-c_0$ is the known cost gap of a feasible action. We also experimentally show the performance of our proposed algorithm in different network structures. |
| title | Cooperative Multi-Agent Constrained Stochastic Linear Bandits |
| topic | Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2410.17382 |