Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.00700 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915447186653184 |
|---|---|
| author | Milosevic, Nikola Müller, Johannes Scherf, Nico |
| author_facet | Milosevic, Nikola Müller, Johannes Scherf, Nico |
| contents | In constrained Markov decision processes, enforcing constraints during training is often thought of as decreasing the final return. Recently, it was shown that constraints can be incorporated directly into the policy geometry, yielding an optimization trajectory close to the central path of a barrier method, which does not compromise final return. Building on this idea, we introduce Central Path Proximal Policy Optimization (C3PO), a simple modification of the PPO loss that produces policy iterates, that stay close to the central path of the constrained optimization problem. Compared to existing on-policy methods, C3PO delivers improved performance with tighter constraint enforcement, suggesting that central path-guided updates offer a promising direction for constrained policy optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_00700 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Central Path Proximal Policy Optimization Milosevic, Nikola Müller, Johannes Scherf, Nico Machine Learning In constrained Markov decision processes, enforcing constraints during training is often thought of as decreasing the final return. Recently, it was shown that constraints can be incorporated directly into the policy geometry, yielding an optimization trajectory close to the central path of a barrier method, which does not compromise final return. Building on this idea, we introduce Central Path Proximal Policy Optimization (C3PO), a simple modification of the PPO loss that produces policy iterates, that stay close to the central path of the constrained optimization problem. Compared to existing on-policy methods, C3PO delivers improved performance with tighter constraint enforcement, suggesting that central path-guided updates offer a promising direction for constrained policy optimization. |
| title | Central Path Proximal Policy Optimization |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.00700 |