Adapting World Models with Latent-State Dynamics Residuals
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915225122373632 |
|---|---|
| author | Lanier, JB Kim, Kyungmin Karamzade, Armin Liu, Yifei Sinha, Ankita He, Kat Corsi, Davide Fox, Roy |
| author_facet | Lanier, JB Kim, Kyungmin Karamzade, Armin Liu, Yifei Sinha, Ankita He, Kat Corsi, Davide Fox, Roy |
| contents | Simulation-to-reality reinforcement learning (RL) faces the critical challenge of reconciling discrepancies between simulated and real-world dynamics, which can severely degrade agent performance. A promising approach involves learning corrections to simulator forward dynamics represented as a residual error function, however this operation is impractical with high-dimensional states such as images. To overcome this, we propose ReDRAW, a latent-state autoregressive world model pretrained in simulation and calibrated to target environments through residual corrections of latent-state dynamics rather than of explicit observed states. Using this adapted world model, ReDRAW enables RL agents to be optimized with imagined rollouts under corrected dynamics and then deployed in the real world. In multiple vision-based MuJoCo domains and a physical robot visual lane-following task, ReDRAW effectively models changes to dynamics and avoids overfitting in low data regimes where traditional transfer methods fail. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_02252 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Adapting World Models with Latent-State Dynamics Residuals Lanier, JB Kim, Kyungmin Karamzade, Armin Liu, Yifei Sinha, Ankita He, Kat Corsi, Davide Fox, Roy Machine Learning Artificial Intelligence Robotics Simulation-to-reality reinforcement learning (RL) faces the critical challenge of reconciling discrepancies between simulated and real-world dynamics, which can severely degrade agent performance. A promising approach involves learning corrections to simulator forward dynamics represented as a residual error function, however this operation is impractical with high-dimensional states such as images. To overcome this, we propose ReDRAW, a latent-state autoregressive world model pretrained in simulation and calibrated to target environments through residual corrections of latent-state dynamics rather than of explicit observed states. Using this adapted world model, ReDRAW enables RL agents to be optimized with imagined rollouts under corrected dynamics and then deployed in the real world. In multiple vision-based MuJoCo domains and a physical robot visual lane-following task, ReDRAW effectively models changes to dynamics and avoids overfitting in low data regimes where traditional transfer methods fail. |
| title | Adapting World Models with Latent-State Dynamics Residuals |
| topic | Machine Learning Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2504.02252 |