Simple Lifelong Learning Machines
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2020
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908474044055552 |
|---|---|
| author | Dey, Jayanta Vogelstein, Joshua T. Helm, Hayden S. LeVine, Will Mehta, Ronak D. Tomita, Tyler M. Xu, Haoyin Geisa, Ali Wang, Qingyang van de Ven, Gido M. Gao, Chenyu Yang, Weiwei Tower, Bryan Larson, Jonathan White, Christopher M. Priebe, Carey E. |
| author_facet | Dey, Jayanta Vogelstein, Joshua T. Helm, Hayden S. LeVine, Will Mehta, Ronak D. Tomita, Tyler M. Xu, Haoyin Geisa, Ali Wang, Qingyang van de Ven, Gido M. Gao, Chenyu Yang, Weiwei Tower, Bryan Larson, Jonathan White, Christopher M. Priebe, Carey E. |
| contents | In lifelong learning, data are used to improve performance not only on the present task, but also on past and future (unencountered) tasks. While typical transfer learning algorithms can improve performance on future tasks, their performance on prior tasks degrades upon learning new tasks (called forgetting). Many recent approaches for continual or lifelong learning have attempted to maintain performance on old tasks given new tasks. But striving to avoid forgetting sets the goal unnecessarily low. The goal of lifelong learning should be to use data to improve performance on both future tasks (forward transfer) and past tasks (backward transfer). In this paper, we show that a simple approach -- representation ensembling -- demonstrates both forward and backward transfer in a variety of simulated and benchmark data scenarios, including tabular, vision (CIFAR-100, 5-dataset, Split Mini-Imagenet, and Food1k), and speech (spoken digit), in contrast to various reference algorithms, which typically failed to transfer either forward or backward, or both. Moreover, our proposed approach can flexibly operate with or without a computational budget. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2004_12908 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | Simple Lifelong Learning Machines Dey, Jayanta Vogelstein, Joshua T. Helm, Hayden S. LeVine, Will Mehta, Ronak D. Tomita, Tyler M. Xu, Haoyin Geisa, Ali Wang, Qingyang van de Ven, Gido M. Gao, Chenyu Yang, Weiwei Tower, Bryan Larson, Jonathan White, Christopher M. Priebe, Carey E. Artificial Intelligence Machine Learning In lifelong learning, data are used to improve performance not only on the present task, but also on past and future (unencountered) tasks. While typical transfer learning algorithms can improve performance on future tasks, their performance on prior tasks degrades upon learning new tasks (called forgetting). Many recent approaches for continual or lifelong learning have attempted to maintain performance on old tasks given new tasks. But striving to avoid forgetting sets the goal unnecessarily low. The goal of lifelong learning should be to use data to improve performance on both future tasks (forward transfer) and past tasks (backward transfer). In this paper, we show that a simple approach -- representation ensembling -- demonstrates both forward and backward transfer in a variety of simulated and benchmark data scenarios, including tabular, vision (CIFAR-100, 5-dataset, Split Mini-Imagenet, and Food1k), and speech (spoken digit), in contrast to various reference algorithms, which typically failed to transfer either forward or backward, or both. Moreover, our proposed approach can flexibly operate with or without a computational budget. |
| title | Simple Lifelong Learning Machines |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2004.12908 |