Epistemic Exploration for Generalizable Planning and Learning in Non-Stationary Settings
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866907988980137984 |
|---|---|
| author | Karia, Rushang Verma, Pulkit Speranzon, Alberto Srivastava, Siddharth |
| author_facet | Karia, Rushang Verma, Pulkit Speranzon, Alberto Srivastava, Siddharth |
| contents | This paper introduces a new approach for continual planning and model learning in relational, non-stationary stochastic environments. Such capabilities are essential for the deployment of sequential decision-making systems in the uncertain and constantly evolving real world. Working in such practical settings with unknown (and non-stationary) transition systems and changing tasks, the proposed framework models gaps in the agent's current state of knowledge and uses them to conduct focused, investigative explorations. Data collected using these explorations is used for learning generalizable probabilistic models for solving the current task despite continual changes in the environment dynamics. Empirical evaluations on several non-stationary benchmark domains show that this approach significantly outperforms planning and RL baselines in terms of sample complexity. Theoretical results show that the system exhibits desirable convergence properties when stationarity holds. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_08145 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Epistemic Exploration for Generalizable Planning and Learning in Non-Stationary Settings Karia, Rushang Verma, Pulkit Speranzon, Alberto Srivastava, Siddharth Artificial Intelligence This paper introduces a new approach for continual planning and model learning in relational, non-stationary stochastic environments. Such capabilities are essential for the deployment of sequential decision-making systems in the uncertain and constantly evolving real world. Working in such practical settings with unknown (and non-stationary) transition systems and changing tasks, the proposed framework models gaps in the agent's current state of knowledge and uses them to conduct focused, investigative explorations. Data collected using these explorations is used for learning generalizable probabilistic models for solving the current task despite continual changes in the environment dynamics. Empirical evaluations on several non-stationary benchmark domains show that this approach significantly outperforms planning and RL baselines in terms of sample complexity. Theoretical results show that the system exhibits desirable convergence properties when stationarity holds. |
| title | Epistemic Exploration for Generalizable Planning and Learning in Non-Stationary Settings |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2402.08145 |