Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.15418 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915858087936000 |
|---|---|
| author | Li, Yueheng Xie, Guangming Lu, Zongqing |
| author_facet | Li, Yueheng Xie, Guangming Lu, Zongqing |
| contents | Reinforcement Learning (RL) in partially observable environments poses significant challenges due to the complexity of learning under uncertainty. While additional information, such as that available in simulations, can enhance training, effectively leveraging it remains an open problem. To address this, we introduce Guided Policy Optimization (GPO), a framework that co-trains a guider and a learner. The guider takes advantage of privileged information while ensuring alignment with the learner's policy that is primarily trained via imitation learning. We theoretically demonstrate that this learning scheme achieves optimality comparable to direct RL, thereby overcoming key limitations inherent in existing approaches. Empirical evaluations show strong performance of GPO across various tasks, including continuous control with partial observability and noise, and memory-based challenges, significantly outperforming existing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_15418 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Guided Policy Optimization under Partial Observability Li, Yueheng Xie, Guangming Lu, Zongqing Machine Learning Artificial Intelligence Robotics Reinforcement Learning (RL) in partially observable environments poses significant challenges due to the complexity of learning under uncertainty. While additional information, such as that available in simulations, can enhance training, effectively leveraging it remains an open problem. To address this, we introduce Guided Policy Optimization (GPO), a framework that co-trains a guider and a learner. The guider takes advantage of privileged information while ensuring alignment with the learner's policy that is primarily trained via imitation learning. We theoretically demonstrate that this learning scheme achieves optimality comparable to direct RL, thereby overcoming key limitations inherent in existing approaches. Empirical evaluations show strong performance of GPO across various tasks, including continuous control with partial observability and noise, and memory-based challenges, significantly outperforming existing methods. |
| title | Guided Policy Optimization under Partial Observability |
| topic | Machine Learning Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2505.15418 |