Optimization over Trained (and Sparse) Neural Networks: A Surrogate within a Surrogate
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911523764436992 |
|---|---|
| author | Pham, Hung Ren, Aiden Tahir, Ibrahim Tong, Jiatai Serra, Thiago |
| author_facet | Pham, Hung Ren, Aiden Tahir, Ibrahim Tong, Jiatai Serra, Thiago |
| contents | In constraint learning, we use a neural network as a surrogate for part of the constraints or of the objective function of an optimization model. However, the tractability of the resulting model is heavily influenced by the size of the neural network used as a surrogate. One way to obtain a more tractable surrogate is by pruning the neural network first. In this work, we consider how to approach the setting in which the neural network is actually a given: how can we solve an optimization model embedding a large and predetermined neural network? We propose surrogating the neural network itself by pruning it, which leads to a sparse and more tractable optimization model, for which we hope to still obtain good solutions with respect to the original neural network. For network verification and function maximization models, that indeed leads to better solutions within a time limit, especially -- and surprisingly -- if we skip the standard retraining step known as finetuning. Hence, a pruned network with worse inference for lack of finetuning can be a better surrogate. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_01985 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Optimization over Trained (and Sparse) Neural Networks: A Surrogate within a Surrogate Pham, Hung Ren, Aiden Tahir, Ibrahim Tong, Jiatai Serra, Thiago Optimization and Control Machine Learning In constraint learning, we use a neural network as a surrogate for part of the constraints or of the objective function of an optimization model. However, the tractability of the resulting model is heavily influenced by the size of the neural network used as a surrogate. One way to obtain a more tractable surrogate is by pruning the neural network first. In this work, we consider how to approach the setting in which the neural network is actually a given: how can we solve an optimization model embedding a large and predetermined neural network? We propose surrogating the neural network itself by pruning it, which leads to a sparse and more tractable optimization model, for which we hope to still obtain good solutions with respect to the original neural network. For network verification and function maximization models, that indeed leads to better solutions within a time limit, especially -- and surprisingly -- if we skip the standard retraining step known as finetuning. Hence, a pruned network with worse inference for lack of finetuning can be a better surrogate. |
| title | Optimization over Trained (and Sparse) Neural Networks: A Surrogate within a Surrogate |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2505.01985 |