Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.18190 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914814820876288 |
|---|---|
| author | Bauer, Johann West, Sheldon Alonso, Eduardo Broom, Mark |
| author_facet | Bauer, Johann West, Sheldon Alonso, Eduardo Broom, Mark |
| contents | We present two variants of a multi-agent reinforcement learning algorithm based on evolutionary game theoretic considerations. The intentional simplicity of one variant enables us to prove results on its relationship to a system of ordinary differential equations of replicator-mutator dynamics type, allowing us to present proofs on the algorithm's convergence conditions in various settings via its ODE counterpart. The more complicated variant enables comparisons to Q-learning based algorithms. We compare both variants experimentally to WoLF-PHC and frequency-adjusted Q-learning on a range of settings, illustrating cases of increasing dimensionality where our variants preserve convergence in contrast to more complicated algorithms. The availability of analytic results provides a degree of transferability of results as compared to purely empirical case studies, illustrating the general utility of a dynamical systems perspective on multi-agent reinforcement learning when addressing questions of convergence and reliable generalisation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_18190 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mutation-Bias Learning in Games Bauer, Johann West, Sheldon Alonso, Eduardo Broom, Mark Machine Learning Multiagent Systems Dynamical Systems Optimization and Control Populations and Evolution 37N40 (Primary) 91A26 (Secondary) We present two variants of a multi-agent reinforcement learning algorithm based on evolutionary game theoretic considerations. The intentional simplicity of one variant enables us to prove results on its relationship to a system of ordinary differential equations of replicator-mutator dynamics type, allowing us to present proofs on the algorithm's convergence conditions in various settings via its ODE counterpart. The more complicated variant enables comparisons to Q-learning based algorithms. We compare both variants experimentally to WoLF-PHC and frequency-adjusted Q-learning on a range of settings, illustrating cases of increasing dimensionality where our variants preserve convergence in contrast to more complicated algorithms. The availability of analytic results provides a degree of transferability of results as compared to purely empirical case studies, illustrating the general utility of a dynamical systems perspective on multi-agent reinforcement learning when addressing questions of convergence and reliable generalisation. |
| title | Mutation-Bias Learning in Games |
| topic | Machine Learning Multiagent Systems Dynamical Systems Optimization and Control Populations and Evolution 37N40 (Primary) 91A26 (Secondary) |
| url | https://arxiv.org/abs/2405.18190 |