Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916810299801600 |
|---|---|
| author | Zhang, Erica Zhang, Fangzhao Pilanci, Mert |
| author_facet | Zhang, Erica Zhang, Fangzhao Pilanci, Mert |
| contents | Active learning methods aim to improve sample complexity in machine learning. In this work, we investigate an active learning scheme via a novel gradient-free cutting-plane training method for ReLU networks of arbitrary depth and develop a convergence theory. We demonstrate, for the first time, that cutting-plane algorithms, traditionally used in linear models, can be extended to deep neural networks despite their nonconvexity and nonlinear decision boundaries. Moreover, this training method induces the first deep active learning scheme known to achieve convergence guarantees, revealing a geometric contraction rate of the feasible set. We exemplify the effectiveness of our proposed active learning method against popular deep active learning baselines via both synthetic data experiments and sentimental classification task on real datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_02145 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes Zhang, Erica Zhang, Fangzhao Pilanci, Mert Machine Learning Optimization and Control Active learning methods aim to improve sample complexity in machine learning. In this work, we investigate an active learning scheme via a novel gradient-free cutting-plane training method for ReLU networks of arbitrary depth and develop a convergence theory. We demonstrate, for the first time, that cutting-plane algorithms, traditionally used in linear models, can be extended to deep neural networks despite their nonconvexity and nonlinear decision boundaries. Moreover, this training method induces the first deep active learning scheme known to achieve convergence guarantees, revealing a geometric contraction rate of the feasible set. We exemplify the effectiveness of our proposed active learning method against popular deep active learning baselines via both synthetic data experiments and sentimental classification task on real datasets. |
| title | Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2410.02145 |