Computational Tradeoffs of Optimization-Based Bound Tightening in ReLU Networks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917578545299456 |
|---|---|
| author | Badilla, Fabian Goycoolea, Marcos Muñoz, Gonzalo Serra, Thiago |
| author_facet | Badilla, Fabian Goycoolea, Marcos Muñoz, Gonzalo Serra, Thiago |
| contents | The use of Mixed-Integer Linear Programming (MILP) models to represent neural networks with Rectified Linear Unit (ReLU) activations has become increasingly widespread in the last decade. This has enabled the use of MILP technology to test-or stress-their behavior, to adversarially improve their training, and to embed them in optimization models leveraging their predictive power. Many of these MILP models rely on activation bounds. That is, bounds on the input values of each neuron. In this work, we explore the tradeoff between the tightness of these bounds and the computational effort of solving the resulting MILP models. We provide guidelines for implementing these models based on the impact of network structure, regularization, and rounding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_16699 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Computational Tradeoffs of Optimization-Based Bound Tightening in ReLU Networks Badilla, Fabian Goycoolea, Marcos Muñoz, Gonzalo Serra, Thiago Optimization and Control Machine Learning The use of Mixed-Integer Linear Programming (MILP) models to represent neural networks with Rectified Linear Unit (ReLU) activations has become increasingly widespread in the last decade. This has enabled the use of MILP technology to test-or stress-their behavior, to adversarially improve their training, and to embed them in optimization models leveraging their predictive power. Many of these MILP models rely on activation bounds. That is, bounds on the input values of each neuron. In this work, we explore the tradeoff between the tightness of these bounds and the computational effort of solving the resulting MILP models. We provide guidelines for implementing these models based on the impact of network structure, regularization, and rounding. |
| title | Computational Tradeoffs of Optimization-Based Bound Tightening in ReLU Networks |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2312.16699 |