Imitation Learning in Discounted Linear MDPs without exploration assumptions
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917755841675264 |
|---|---|
| author | Viano, Luca Skoulakis, Stratis Cevher, Volkan |
| author_facet | Viano, Luca Skoulakis, Stratis Cevher, Volkan |
| contents | We present a new algorithm for imitation learning in infinite horizon linear MDPs dubbed ILARL which greatly improves the bound on the number of trajectories that the learner needs to sample from the environment. In particular, we remove exploration assumptions required in previous works and we improve the dependence on the desired accuracy $ε$ from $\mathcal{O}(ε^{-5})$ to $\mathcal{O}(ε^{-4})$. Our result relies on a connection between imitation learning and online learning in MDPs with adversarial losses. For the latter setting, we present the first result for infinite horizon linear MDP which may be of independent interest. Moreover, we are able to provide a strengthen result for the finite horizon case where we achieve $\mathcal{O}(ε^{-2})$. Numerical experiments with linear function approximation shows that ILARL outperforms other commonly used algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_02181 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Imitation Learning in Discounted Linear MDPs without exploration assumptions Viano, Luca Skoulakis, Stratis Cevher, Volkan Machine Learning We present a new algorithm for imitation learning in infinite horizon linear MDPs dubbed ILARL which greatly improves the bound on the number of trajectories that the learner needs to sample from the environment. In particular, we remove exploration assumptions required in previous works and we improve the dependence on the desired accuracy $ε$ from $\mathcal{O}(ε^{-5})$ to $\mathcal{O}(ε^{-4})$. Our result relies on a connection between imitation learning and online learning in MDPs with adversarial losses. For the latter setting, we present the first result for infinite horizon linear MDP which may be of independent interest. Moreover, we are able to provide a strengthen result for the finite horizon case where we achieve $\mathcal{O}(ε^{-2})$. Numerical experiments with linear function approximation shows that ILARL outperforms other commonly used algorithms. |
| title | Imitation Learning in Discounted Linear MDPs without exploration assumptions |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2405.02181 |