Hybrid Random Concentrated Optimization Without Convexity Assumption
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915330089025536 |
|---|---|
| author | Bertrand, Pierre Broniatowski, Michel Stummer, Wolfgang |
| author_facet | Bertrand, Pierre Broniatowski, Michel Stummer, Wolfgang |
| contents | We propose a new random method to minimize deterministic continuous functions over subsets $\mathcal{S}$ of high-dimensional space $\mathbb{R}^K$ without assuming convexity. Our procedure alternates between a Global Search (GS) regime to identify candidates and a Concentrated Search (CS) regime to improve an eligible candidate in the constraint set $\mathcal{S}$. Beyond the alternation between those completely different regimes, the originality of our approach lies in leveraging high dimensionality. We demonstrate rigorous concentration properties under the $CS$ regime. In parallel, we also show that $GS$ reaches any point in $\mathcal{S}$ in finite time. Finally, we demonstrate the relevance of our new method by giving two concrete applications. The first deals with the reduction of the $\ell_{1}-$norm of a LASSO solution. Secondly, we compress a neural network by pruning weights while maintaining performance; our approach achieves significant weight reduction with minimal performance loss, offering an effective solution for network optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_23864 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Hybrid Random Concentrated Optimization Without Convexity Assumption Bertrand, Pierre Broniatowski, Michel Stummer, Wolfgang Data Analysis, Statistics and Probability We propose a new random method to minimize deterministic continuous functions over subsets $\mathcal{S}$ of high-dimensional space $\mathbb{R}^K$ without assuming convexity. Our procedure alternates between a Global Search (GS) regime to identify candidates and a Concentrated Search (CS) regime to improve an eligible candidate in the constraint set $\mathcal{S}$. Beyond the alternation between those completely different regimes, the originality of our approach lies in leveraging high dimensionality. We demonstrate rigorous concentration properties under the $CS$ regime. In parallel, we also show that $GS$ reaches any point in $\mathcal{S}$ in finite time. Finally, we demonstrate the relevance of our new method by giving two concrete applications. The first deals with the reduction of the $\ell_{1}-$norm of a LASSO solution. Secondly, we compress a neural network by pruning weights while maintaining performance; our approach achieves significant weight reduction with minimal performance loss, offering an effective solution for network optimization. |
| title | Hybrid Random Concentrated Optimization Without Convexity Assumption |
| topic | Data Analysis, Statistics and Probability |
| url | https://arxiv.org/abs/2503.23864 |