Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.19523 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929658998554624 |
|---|---|
| author | Wang, Cheems Lv, Yiqin Mao, Yixiu Qu, Yun Xu, Yi Ji, Xiangyang |
| author_facet | Wang, Cheems Lv, Yiqin Mao, Yixiu Qu, Yun Xu, Yi Ji, Xiangyang |
| contents | Meta-learning is a practical learning paradigm to transfer skills across tasks from a few examples. Nevertheless, the existence of task distribution shifts tends to weaken meta-learners' generalization capability, particularly when the training task distribution is naively hand-crafted or based on simple priors that fail to cover critical scenarios sufficiently. Here, we consider explicitly generative modeling task distributions placed over task identifiers and propose robustifying fast adaptation from adversarial training. Our approach, which can be interpreted as a model of a Stackelberg game, not only uncovers the task structure during problem-solving from an explicit generative model but also theoretically increases the adaptation robustness in worst cases. This work has practical implications, particularly in dealing with task distribution shifts in meta-learning, and contributes to theoretical insights in the field. Our method demonstrates its robustness in the presence of task subpopulation shifts and improved performance over SOTA baselines in extensive experiments. The code is available at the project site https://sites.google.com/view/ar-metalearn. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_19523 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation Wang, Cheems Lv, Yiqin Mao, Yixiu Qu, Yun Xu, Yi Ji, Xiangyang Machine Learning Meta-learning is a practical learning paradigm to transfer skills across tasks from a few examples. Nevertheless, the existence of task distribution shifts tends to weaken meta-learners' generalization capability, particularly when the training task distribution is naively hand-crafted or based on simple priors that fail to cover critical scenarios sufficiently. Here, we consider explicitly generative modeling task distributions placed over task identifiers and propose robustifying fast adaptation from adversarial training. Our approach, which can be interpreted as a model of a Stackelberg game, not only uncovers the task structure during problem-solving from an explicit generative model but also theoretically increases the adaptation robustness in worst cases. This work has practical implications, particularly in dealing with task distribution shifts in meta-learning, and contributes to theoretical insights in the field. Our method demonstrates its robustness in the presence of task subpopulation shifts and improved performance over SOTA baselines in extensive experiments. The code is available at the project site https://sites.google.com/view/ar-metalearn. |
| title | Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2407.19523 |