Model-Based Reinforcement Learning for Control under Time-Varying Dynamics
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910098163499008 |
|---|---|
| author | Iten, Klemens Lee, Bruce Li, Chenhao Treven, Lenart Krause, Andreas Sukhija, Bhavya |
| author_facet | Iten, Klemens Lee, Bruce Li, Chenhao Treven, Lenart Krause, Andreas Sukhija, Bhavya |
| contents | Learning-based control methods typically assume stationary system dynamics, an assumption often violated in real-world systems due to drift, wear, or changing operating conditions. We study reinforcement learning for control under time-varying dynamics. We consider a continual model-based reinforcement learning setting in which an agent repeatedly learns and controls a dynamical system whose transition dynamics evolve across episodes. We analyze the problem using Gaussian process dynamics models under frequentist variation-budget assumptions. Our analysis shows that persistent non-stationarity requires explicitly limiting the influence of outdated data to maintain calibrated uncertainty and meaningful dynamic regret guarantees. Motivated by these insights, we propose a practical optimistic model-based reinforcement learning algorithm with adaptive data buffer mechanisms and demonstrate improved performance on continuous control benchmarks with non-stationary dynamics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_02260 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Model-Based Reinforcement Learning for Control under Time-Varying Dynamics Iten, Klemens Lee, Bruce Li, Chenhao Treven, Lenart Krause, Andreas Sukhija, Bhavya Machine Learning Robotics Learning-based control methods typically assume stationary system dynamics, an assumption often violated in real-world systems due to drift, wear, or changing operating conditions. We study reinforcement learning for control under time-varying dynamics. We consider a continual model-based reinforcement learning setting in which an agent repeatedly learns and controls a dynamical system whose transition dynamics evolve across episodes. We analyze the problem using Gaussian process dynamics models under frequentist variation-budget assumptions. Our analysis shows that persistent non-stationarity requires explicitly limiting the influence of outdated data to maintain calibrated uncertainty and meaningful dynamic regret guarantees. Motivated by these insights, we propose a practical optimistic model-based reinforcement learning algorithm with adaptive data buffer mechanisms and demonstrate improved performance on continuous control benchmarks with non-stationary dynamics. |
| title | Model-Based Reinforcement Learning for Control under Time-Varying Dynamics |
| topic | Machine Learning Robotics |
| url | https://arxiv.org/abs/2604.02260 |