Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912649887875072 |
|---|---|
| author | Obando-Ceron, Johan Mayor, Walter Lavoie, Samuel Fujimoto, Scott Courville, Aaron Castro, Pablo Samuel |
| author_facet | Obando-Ceron, Johan Mayor, Walter Lavoie, Samuel Fujimoto, Scott Courville, Aaron Castro, Pablo Samuel |
| contents | Recent works have proposed accelerating the wall-clock training time of actor-critic methods via the use of large-scale environment parallelization; unfortunately, these can sometimes still require large number of environment interactions to achieve a desired level of performance. Noting that well-structured representations can improve the generalization and sample efficiency of deep reinforcement learning (RL) agents, we propose the use of simplicial embeddings: lightweight representation layers that constrain embeddings to simplicial structures. This geometric inductive bias results in sparse and discrete features that stabilize critic bootstrapping and strengthen policy gradients. When applied to FastTD3, FastSAC, and PPO, simplicial embeddings consistently improve sample efficiency and final performance across a variety of continuous- and discrete-control environments, without any loss in runtime speed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_13704 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agents Obando-Ceron, Johan Mayor, Walter Lavoie, Samuel Fujimoto, Scott Courville, Aaron Castro, Pablo Samuel Machine Learning Artificial Intelligence Robotics Recent works have proposed accelerating the wall-clock training time of actor-critic methods via the use of large-scale environment parallelization; unfortunately, these can sometimes still require large number of environment interactions to achieve a desired level of performance. Noting that well-structured representations can improve the generalization and sample efficiency of deep reinforcement learning (RL) agents, we propose the use of simplicial embeddings: lightweight representation layers that constrain embeddings to simplicial structures. This geometric inductive bias results in sparse and discrete features that stabilize critic bootstrapping and strengthen policy gradients. When applied to FastTD3, FastSAC, and PPO, simplicial embeddings consistently improve sample efficiency and final performance across a variety of continuous- and discrete-control environments, without any loss in runtime speed. |
| title | Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agents |
| topic | Machine Learning Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2510.13704 |