When Deep Learning Meets Polyhedral Theory: A Survey
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908536007557120 |
|---|---|
| author | Huchette, Joey Muñoz, Gonzalo Serra, Thiago Tsay, Calvin |
| author_facet | Huchette, Joey Muñoz, Gonzalo Serra, Thiago Tsay, Calvin |
| contents | In the past decade, deep learning became the prevalent methodology for predictive modeling thanks to the remarkable accuracy of deep neural networks in tasks such as computer vision and natural language processing. Meanwhile, the structure of neural networks converged back to simpler representations based on piecewise constant and piecewise linear functions such as the Rectified Linear Unit (ReLU), which became the most commonly used type of activation function in neural networks. That made certain types of network structure $\unicode{x2014}$such as the typical fully-connected feedforward neural network$\unicode{x2014}$ amenable to analysis through polyhedral theory and to the application of methodologies such as Linear Programming (LP) and Mixed-Integer Linear Programming (MILP) for a variety of purposes. In this paper, we survey the main topics emerging from this fast-paced area of work, which bring a fresh perspective to understanding neural networks in more detail as well as to applying linear optimization techniques to train, verify, and reduce the size of such networks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_00241 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | When Deep Learning Meets Polyhedral Theory: A Survey Huchette, Joey Muñoz, Gonzalo Serra, Thiago Tsay, Calvin Optimization and Control Machine Learning In the past decade, deep learning became the prevalent methodology for predictive modeling thanks to the remarkable accuracy of deep neural networks in tasks such as computer vision and natural language processing. Meanwhile, the structure of neural networks converged back to simpler representations based on piecewise constant and piecewise linear functions such as the Rectified Linear Unit (ReLU), which became the most commonly used type of activation function in neural networks. That made certain types of network structure $\unicode{x2014}$such as the typical fully-connected feedforward neural network$\unicode{x2014}$ amenable to analysis through polyhedral theory and to the application of methodologies such as Linear Programming (LP) and Mixed-Integer Linear Programming (MILP) for a variety of purposes. In this paper, we survey the main topics emerging from this fast-paced area of work, which bring a fresh perspective to understanding neural networks in more detail as well as to applying linear optimization techniques to train, verify, and reduce the size of such networks. |
| title | When Deep Learning Meets Polyhedral Theory: A Survey |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2305.00241 |