From Sparse to Dense: Toddler-inspired Reward Transition in Goal-Oriented Reinforcement Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917906100518912 |
|---|---|
| author | Park, Junseok Yang, Hyeonseo Lee, Min Whoo Choi, Won-Seok Lee, Minsu Zhang, Byoung-Tak |
| author_facet | Park, Junseok Yang, Hyeonseo Lee, Min Whoo Choi, Won-Seok Lee, Minsu Zhang, Byoung-Tak |
| contents | Reinforcement learning (RL) agents often face challenges in balancing exploration and exploitation, particularly in environments where sparse or dense rewards bias learning. Biological systems, such as human toddlers, naturally navigate this balance by transitioning from free exploration with sparse rewards to goal-directed behavior guided by increasingly dense rewards. Inspired by this natural progression, we investigate the Toddler-Inspired Reward Transition in goal-oriented RL tasks. Our study focuses on transitioning from sparse to potential-based dense (S2D) rewards while preserving optimal strategies. Through experiments on dynamic robotic arm manipulation and egocentric 3D navigation tasks, we demonstrate that effective S2D reward transitions significantly enhance learning performance and sample efficiency. Additionally, using a Cross-Density Visualizer, we show that S2D transitions smooth the policy loss landscape, resulting in wider minima that improve generalization in RL models. In addition, we reinterpret Tolman's maze experiments, underscoring the critical role of early free exploratory learning in the context of S2D rewards. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_17842 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Sparse to Dense: Toddler-inspired Reward Transition in Goal-Oriented Reinforcement Learning Park, Junseok Yang, Hyeonseo Lee, Min Whoo Choi, Won-Seok Lee, Minsu Zhang, Byoung-Tak Machine Learning Artificial Intelligence Robotics 68T05, 68T20, 91E40 Reinforcement learning (RL) agents often face challenges in balancing exploration and exploitation, particularly in environments where sparse or dense rewards bias learning. Biological systems, such as human toddlers, naturally navigate this balance by transitioning from free exploration with sparse rewards to goal-directed behavior guided by increasingly dense rewards. Inspired by this natural progression, we investigate the Toddler-Inspired Reward Transition in goal-oriented RL tasks. Our study focuses on transitioning from sparse to potential-based dense (S2D) rewards while preserving optimal strategies. Through experiments on dynamic robotic arm manipulation and egocentric 3D navigation tasks, we demonstrate that effective S2D reward transitions significantly enhance learning performance and sample efficiency. Additionally, using a Cross-Density Visualizer, we show that S2D transitions smooth the policy loss landscape, resulting in wider minima that improve generalization in RL models. In addition, we reinterpret Tolman's maze experiments, underscoring the critical role of early free exploratory learning in the context of S2D rewards. |
| title | From Sparse to Dense: Toddler-inspired Reward Transition in Goal-Oriented Reinforcement Learning |
| topic | Machine Learning Artificial Intelligence Robotics 68T05, 68T20, 91E40 |
| url | https://arxiv.org/abs/2501.17842 |