Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2411.07832 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910695744864256 |
|---|---|
| author | Han, Chao Basu, Debabrota Mangan, Michael Vasilaki, Eleni Gilra, Aditya |
| author_facet | Han, Chao Basu, Debabrota Mangan, Michael Vasilaki, Eleni Gilra, Aditya |
| contents | Learning representations of underlying environmental dynamics from partial observations is a critical challenge in machine learning. In the context of Partially Observable Markov Decision Processes (POMDPs), state representations are often inferred from the history of past observations and actions. We demonstrate that incorporating future information is essential to accurately capture causal dynamics and enhance state representations. To address this, we introduce a Dynamical Variational Auto-Encoder (DVAE) designed to learn causal Markovian dynamics from offline trajectories in a POMDP. Our method employs an extended hindsight framework that integrates past, current, and multi-step future information within a factored-POMDP setting. Empirical results reveal that this approach uncovers the causal graph governing hidden state transitions more effectively than history-based and typical hindsight-based models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_07832 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Dynamical-VAE-based Hindsight to Learn the Causal Dynamics of Factored-POMDPs Han, Chao Basu, Debabrota Mangan, Michael Vasilaki, Eleni Gilra, Aditya Machine Learning Learning representations of underlying environmental dynamics from partial observations is a critical challenge in machine learning. In the context of Partially Observable Markov Decision Processes (POMDPs), state representations are often inferred from the history of past observations and actions. We demonstrate that incorporating future information is essential to accurately capture causal dynamics and enhance state representations. To address this, we introduce a Dynamical Variational Auto-Encoder (DVAE) designed to learn causal Markovian dynamics from offline trajectories in a POMDP. Our method employs an extended hindsight framework that integrates past, current, and multi-step future information within a factored-POMDP setting. Empirical results reveal that this approach uncovers the causal graph governing hidden state transitions more effectively than history-based and typical hindsight-based models. |
| title | Dynamical-VAE-based Hindsight to Learn the Causal Dynamics of Factored-POMDPs |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2411.07832 |