A Theoretical Justification for Asymmetric Actor-Critic Algorithms
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915483599503360 |
|---|---|
| author | Lambrechts, Gaspard Ernst, Damien Mahajan, Aditya |
| author_facet | Lambrechts, Gaspard Ernst, Damien Mahajan, Aditya |
| contents | In reinforcement learning for partially observable environments, many successful algorithms have been developed within the asymmetric learning paradigm. This paradigm leverages additional state information available at training time for faster learning. Although the proposed learning objectives are usually theoretically sound, these methods still lack a precise theoretical justification for their potential benefits. We propose such a justification for asymmetric actor-critic algorithms with linear function approximators by adapting a finite-time convergence analysis to this setting. The resulting finite-time bound reveals that the asymmetric critic eliminates error terms arising from aliasing in the agent state. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_19116 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Theoretical Justification for Asymmetric Actor-Critic Algorithms Lambrechts, Gaspard Ernst, Damien Mahajan, Aditya Machine Learning In reinforcement learning for partially observable environments, many successful algorithms have been developed within the asymmetric learning paradigm. This paradigm leverages additional state information available at training time for faster learning. Although the proposed learning objectives are usually theoretically sound, these methods still lack a precise theoretical justification for their potential benefits. We propose such a justification for asymmetric actor-critic algorithms with linear function approximators by adapting a finite-time convergence analysis to this setting. The resulting finite-time bound reveals that the asymmetric critic eliminates error terms arising from aliasing in the agent state. |
| title | A Theoretical Justification for Asymmetric Actor-Critic Algorithms |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2501.19116 |