An Optimisation Framework for Unsupervised Environment Design
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913933080657920 |
|---|---|
| author | Monette, Nathan Letcher, Alistair Beukman, Michael Jackson, Matthew T. Rutherford, Alexander Goldie, Alexander D. Foerster, Jakob N. |
| author_facet | Monette, Nathan Letcher, Alistair Beukman, Michael Jackson, Matthew T. Rutherford, Alexander Goldie, Alexander D. Foerster, Jakob N. |
| contents | For reinforcement learning agents to be deployed in high-risk settings, they must achieve a high level of robustness to unfamiliar scenarios. One method for improving robustness is unsupervised environment design (UED), a suite of methods aiming to maximise an agent's generalisability across configurations of an environment. In this work, we study UED from an optimisation perspective, providing stronger theoretical guarantees for practical settings than prior work. Whereas previous methods relied on guarantees if they reach convergence, our framework employs a nonconvex-strongly-concave objective for which we provide a provably convergent algorithm in the zero-sum setting. We empirically verify the efficacy of our method, outperforming prior methods in a number of environments with varying difficulties. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_20659 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An Optimisation Framework for Unsupervised Environment Design Monette, Nathan Letcher, Alistair Beukman, Michael Jackson, Matthew T. Rutherford, Alexander Goldie, Alexander D. Foerster, Jakob N. Machine Learning For reinforcement learning agents to be deployed in high-risk settings, they must achieve a high level of robustness to unfamiliar scenarios. One method for improving robustness is unsupervised environment design (UED), a suite of methods aiming to maximise an agent's generalisability across configurations of an environment. In this work, we study UED from an optimisation perspective, providing stronger theoretical guarantees for practical settings than prior work. Whereas previous methods relied on guarantees if they reach convergence, our framework employs a nonconvex-strongly-concave objective for which we provide a provably convergent algorithm in the zero-sum setting. We empirically verify the efficacy of our method, outperforming prior methods in a number of environments with varying difficulties. |
| title | An Optimisation Framework for Unsupervised Environment Design |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.20659 |