A Survey of Lottery Ticket Hypothesis
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917611828150272 |
|---|---|
| author | Liu, Bohan Zhang, Zijie He, Peixiong Wang, Zhensen Xiao, Yang Ye, Ruimeng Zhou, Yang Ku, Wei-Shinn Hui, Bo |
| author_facet | Liu, Bohan Zhang, Zijie He, Peixiong Wang, Zhensen Xiao, Yang Ye, Ruimeng Zhou, Yang Ku, Wei-Shinn Hui, Bo |
| contents | The Lottery Ticket Hypothesis (LTH) states that a dense neural network model contains a highly sparse subnetwork (i.e., winning tickets) that can achieve even better performance than the original model when trained in isolation. While LTH has been proved both empirically and theoretically in many works, there still are some open issues, such as efficiency and scalability, to be addressed. Also, the lack of open-source frameworks and consensual experimental setting poses a challenge to future research on LTH. We, for the first time, examine previous research and studies on LTH from different perspectives. We also discuss issues in existing works and list potential directions for further exploration. This survey aims to provide an in-depth look at the state of LTH and develop a duly maintained platform to conduct experiments and compare with the most updated baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_04861 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Survey of Lottery Ticket Hypothesis Liu, Bohan Zhang, Zijie He, Peixiong Wang, Zhensen Xiao, Yang Ye, Ruimeng Zhou, Yang Ku, Wei-Shinn Hui, Bo Machine Learning Neural and Evolutionary Computing The Lottery Ticket Hypothesis (LTH) states that a dense neural network model contains a highly sparse subnetwork (i.e., winning tickets) that can achieve even better performance than the original model when trained in isolation. While LTH has been proved both empirically and theoretically in many works, there still are some open issues, such as efficiency and scalability, to be addressed. Also, the lack of open-source frameworks and consensual experimental setting poses a challenge to future research on LTH. We, for the first time, examine previous research and studies on LTH from different perspectives. We also discuss issues in existing works and list potential directions for further exploration. This survey aims to provide an in-depth look at the state of LTH and develop a duly maintained platform to conduct experiments and compare with the most updated baselines. |
| title | A Survey of Lottery Ticket Hypothesis |
| topic | Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2403.04861 |