Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.23075 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917235314917376 |
|---|---|
| author | Bian, Yuexin Feng, Jie Wang, Tao Li, Yijiang Gao, Sicun Shi, Yuanyuan |
| author_facet | Bian, Yuexin Feng, Jie Wang, Tao Li, Yijiang Gao, Sicun Shi, Yuanyuan |
| contents | On-policy deep reinforcement learning remains a dominant paradigm for continuous control, yet standard implementations rely on Gaussian actors and relatively shallow MLP policies, often leading to brittle optimization when gradients are noisy and policy updates must be conservative. In this paper, we revisit policy representation as a first-class design choice for on-policy optimization. We study discretized categorical actors that represent each action dimension with a distribution over bins, yielding a policy objective that resembles a cross-entropy loss. Building on architectural advances from supervised learning, we further propose regularized actor networks, while keeping critic design fixed. Our results show that simply replacing the standard actor network with our discretized regularized actor yields consistent gains and achieve the state-of-the-art performance across diverse continuous-control benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_23075 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | RN-D: Discretized Categorical Actors with Regularized Networks for On-Policy Reinforcement Learning Bian, Yuexin Feng, Jie Wang, Tao Li, Yijiang Gao, Sicun Shi, Yuanyuan Machine Learning Robotics On-policy deep reinforcement learning remains a dominant paradigm for continuous control, yet standard implementations rely on Gaussian actors and relatively shallow MLP policies, often leading to brittle optimization when gradients are noisy and policy updates must be conservative. In this paper, we revisit policy representation as a first-class design choice for on-policy optimization. We study discretized categorical actors that represent each action dimension with a distribution over bins, yielding a policy objective that resembles a cross-entropy loss. Building on architectural advances from supervised learning, we further propose regularized actor networks, while keeping critic design fixed. Our results show that simply replacing the standard actor network with our discretized regularized actor yields consistent gains and achieve the state-of-the-art performance across diverse continuous-control benchmarks. |
| title | RN-D: Discretized Categorical Actors with Regularized Networks for On-Policy Reinforcement Learning |
| topic | Machine Learning Robotics |
| url | https://arxiv.org/abs/2601.23075 |