Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929575617888256 |
|---|---|
| author | Ganesh, Swetha Chen, Jiayu Thoppe, Gugan Aggarwal, Vaneet |
| author_facet | Ganesh, Swetha Chen, Jiayu Thoppe, Gugan Aggarwal, Vaneet |
| contents | Federated Reinforcement Learning (FRL) allows multiple agents to collaboratively build a decision making policy without sharing raw trajectories. However, if a small fraction of these agents are adversarial, it can lead to catastrophic results. We propose a policy gradient based approach that is robust to adversarial agents which can send arbitrary values to the server. Under this setting, our results form the first global convergence guarantees with general parametrization. These results demonstrate resilience with adversaries, while achieving optimal sample complexity of order $\tilde{\mathcal{O}}\left( \frac{1}{Nε^2} \left( 1+ \frac{f^2}{N}\right)\right)$, where $N$ is the total number of agents and $f<N/2$ is the number of adversarial agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_09940 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries Ganesh, Swetha Chen, Jiayu Thoppe, Gugan Aggarwal, Vaneet Machine Learning Artificial Intelligence Optimization and Control Federated Reinforcement Learning (FRL) allows multiple agents to collaboratively build a decision making policy without sharing raw trajectories. However, if a small fraction of these agents are adversarial, it can lead to catastrophic results. We propose a policy gradient based approach that is robust to adversarial agents which can send arbitrary values to the server. Under this setting, our results form the first global convergence guarantees with general parametrization. These results demonstrate resilience with adversaries, while achieving optimal sample complexity of order $\tilde{\mathcal{O}}\left( \frac{1}{Nε^2} \left( 1+ \frac{f^2}{N}\right)\right)$, where $N$ is the total number of agents and $f<N/2$ is the number of adversarial agents. |
| title | Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries |
| topic | Machine Learning Artificial Intelligence Optimization and Control |
| url | https://arxiv.org/abs/2403.09940 |