Efficient Dynamics Modeling in Interactive Environments with Koopman Theory
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914792098234368 |
|---|---|
| author | Mondal, Arnab Kumar Panigrahi, Siba Smarak Rajeswar, Sai Siddiqi, Kaleem Ravanbakhsh, Siamak |
| author_facet | Mondal, Arnab Kumar Panigrahi, Siba Smarak Rajeswar, Sai Siddiqi, Kaleem Ravanbakhsh, Siamak |
| contents | The accurate modeling of dynamics in interactive environments is critical for successful long-range prediction. Such a capability could advance Reinforcement Learning (RL) and Planning algorithms, but achieving it is challenging. Inaccuracies in model estimates can compound, resulting in increased errors over long horizons. We approach this problem from the lens of Koopman theory, where the nonlinear dynamics of the environment can be linearized in a high-dimensional latent space. This allows us to efficiently parallelize the sequential problem of long-range prediction using convolution while accounting for the agent's action at every time step. Our approach also enables stability analysis and better control over gradients through time. Taken together, these advantages result in significant improvement over the existing approaches, both in the efficiency and the accuracy of modeling dynamics over extended horizons. We also show that this model can be easily incorporated into dynamics modeling for model-based planning and model-free RL and report promising experimental results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_11941 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Efficient Dynamics Modeling in Interactive Environments with Koopman Theory Mondal, Arnab Kumar Panigrahi, Siba Smarak Rajeswar, Sai Siddiqi, Kaleem Ravanbakhsh, Siamak Machine Learning Artificial Intelligence The accurate modeling of dynamics in interactive environments is critical for successful long-range prediction. Such a capability could advance Reinforcement Learning (RL) and Planning algorithms, but achieving it is challenging. Inaccuracies in model estimates can compound, resulting in increased errors over long horizons. We approach this problem from the lens of Koopman theory, where the nonlinear dynamics of the environment can be linearized in a high-dimensional latent space. This allows us to efficiently parallelize the sequential problem of long-range prediction using convolution while accounting for the agent's action at every time step. Our approach also enables stability analysis and better control over gradients through time. Taken together, these advantages result in significant improvement over the existing approaches, both in the efficiency and the accuracy of modeling dynamics over extended horizons. We also show that this model can be easily incorporated into dynamics modeling for model-based planning and model-free RL and report promising experimental results. |
| title | Efficient Dynamics Modeling in Interactive Environments with Koopman Theory |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2306.11941 |