Adapting the Behavior of Reinforcement Learning Agents to Changing Action Spaces and Reward Functions
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915759662301184 |
|---|---|
| author | de la Rosa, Raul Dusparic, Ivana Cardozo, Nicolas |
| author_facet | de la Rosa, Raul Dusparic, Ivana Cardozo, Nicolas |
| contents | Reinforcement Learning (RL) agents often struggle in real-world applications where environmental conditions are non-stationary, particularly when reward functions shift or the available action space expands. This paper introduces MORPHIN, a self-adaptive Q-learning framework that enables on-the-fly adaptation without full retraining. By integrating concept drift detection with dynamic adjustments to learning and exploration hyperparameters, MORPHIN adapts agents to changes in both the reward function and on-the-fly expansions of the agent's action space, while preserving prior policy knowledge to prevent catastrophic forgetting. We validate our approach using a Gridworld benchmark and a traffic signal control simulation. The results demonstrate that MORPHIN achieves superior convergence speed and continuous adaptation compared to a standard Q-learning baseline, improving learning efficiency by up to 1.7x. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_20714 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Adapting the Behavior of Reinforcement Learning Agents to Changing Action Spaces and Reward Functions de la Rosa, Raul Dusparic, Ivana Cardozo, Nicolas Machine Learning Artificial Intelligence Reinforcement Learning (RL) agents often struggle in real-world applications where environmental conditions are non-stationary, particularly when reward functions shift or the available action space expands. This paper introduces MORPHIN, a self-adaptive Q-learning framework that enables on-the-fly adaptation without full retraining. By integrating concept drift detection with dynamic adjustments to learning and exploration hyperparameters, MORPHIN adapts agents to changes in both the reward function and on-the-fly expansions of the agent's action space, while preserving prior policy knowledge to prevent catastrophic forgetting. We validate our approach using a Gridworld benchmark and a traffic signal control simulation. The results demonstrate that MORPHIN achieves superior convergence speed and continuous adaptation compared to a standard Q-learning baseline, improving learning efficiency by up to 1.7x. |
| title | Adapting the Behavior of Reinforcement Learning Agents to Changing Action Spaces and Reward Functions |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2601.20714 |