Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.05619 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915777824686080 |
|---|---|
| author | Mohamad, Mohamad Ponzio, Francesco Descombes, Xavier |
| author_facet | Mohamad, Mohamad Ponzio, Francesco Descombes, Xavier |
| contents | Mode-dependent architectural components (layers that behave differently during training and evaluation, such as Batch Normalization or dropout) are commonly used in visual reinforcement learning but can destabilize on-policy optimization. We show that in Proximal Policy Optimization (PPO), discrepancies between training and evaluation behavior induced by Batch Normalization lead to policy mismatch, distributional drift, and reward collapse. We propose Mode-Dependent Rectification (MDR), a lightweight dual-phase training procedure that stabilizes PPO under mode-dependent layers without architectural changes. Experiments across procedurally generated games and real-world patch-localization tasks demonstrate that MDR consistently improves stability and performance, and extends naturally to other mode-dependent layers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_05619 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Mode-Dependent Rectification for Stable PPO Training Mohamad, Mohamad Ponzio, Francesco Descombes, Xavier Machine Learning Artificial Intelligence Mode-dependent architectural components (layers that behave differently during training and evaluation, such as Batch Normalization or dropout) are commonly used in visual reinforcement learning but can destabilize on-policy optimization. We show that in Proximal Policy Optimization (PPO), discrepancies between training and evaluation behavior induced by Batch Normalization lead to policy mismatch, distributional drift, and reward collapse. We propose Mode-Dependent Rectification (MDR), a lightweight dual-phase training procedure that stabilizes PPO under mode-dependent layers without architectural changes. Experiments across procedurally generated games and real-world patch-localization tasks demonstrate that MDR consistently improves stability and performance, and extends naturally to other mode-dependent layers. |
| title | Mode-Dependent Rectification for Stable PPO Training |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2602.05619 |