An investigation of stochastic trust-region based algorithms for finite-sum minimization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913322606002176 |
|---|---|
| author | Bellavia, Stefania Morini, Benedetta Rebegoldi, Simone |
| author_facet | Bellavia, Stefania Morini, Benedetta Rebegoldi, Simone |
| contents | This work elaborates on the TRust-region-ish (TRish) algorithm, a stochastic optimization method for finite-sum minimization problems proposed by Curtis et al. in [Curtis2019, Curtis2022]. A theoretical analysis that complements the results in the literature is presented, and the issue of tuning the involved hyper-parameters is investigated. Our study also focuses on a practical version of the method, which computes the stochastic gradient by means of the inner product test and the orthogonality test proposed by Bollapragada et al. in [Bollapragada2018]. It is shown experimentally that this implementation improves the performance of TRish and reduces its sensitivity to the choice of the hyper-parameters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_13382 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An investigation of stochastic trust-region based algorithms for finite-sum minimization Bellavia, Stefania Morini, Benedetta Rebegoldi, Simone Optimization and Control 65K05, 90C30, 90C15 This work elaborates on the TRust-region-ish (TRish) algorithm, a stochastic optimization method for finite-sum minimization problems proposed by Curtis et al. in [Curtis2019, Curtis2022]. A theoretical analysis that complements the results in the literature is presented, and the issue of tuning the involved hyper-parameters is investigated. Our study also focuses on a practical version of the method, which computes the stochastic gradient by means of the inner product test and the orthogonality test proposed by Bollapragada et al. in [Bollapragada2018]. It is shown experimentally that this implementation improves the performance of TRish and reduces its sensitivity to the choice of the hyper-parameters. |
| title | An investigation of stochastic trust-region based algorithms for finite-sum minimization |
| topic | Optimization and Control 65K05, 90C30, 90C15 |
| url | https://arxiv.org/abs/2404.13382 |