Online estimation of the inverse of the Hessian for stochastic optimization with application to universal stochastic Newton algorithms
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912265981132800 |
|---|---|
| author | Godichon-Baggioni, Antoine Lu, Wei Portier, Bruno |
| author_facet | Godichon-Baggioni, Antoine Lu, Wei Portier, Bruno |
| contents | This paper addresses second-order stochastic optimization for estimating the minimizer of a convex function written as an expectation. A direct recursive estimation technique for the inverse Hessian matrix using a Robbins-Monro procedure is introduced. This approach enables to drastically reduces computational complexity. Above all, it allows to develop universal stochastic Newton methods and investigate the asymptotic efficiency of the proposed approach. This work so expands the application scope of secondorder algorithms in stochastic optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_10923 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Online estimation of the inverse of the Hessian for stochastic optimization with application to universal stochastic Newton algorithms Godichon-Baggioni, Antoine Lu, Wei Portier, Bruno Optimization and Control Machine Learning This paper addresses second-order stochastic optimization for estimating the minimizer of a convex function written as an expectation. A direct recursive estimation technique for the inverse Hessian matrix using a Robbins-Monro procedure is introduced. This approach enables to drastically reduces computational complexity. Above all, it allows to develop universal stochastic Newton methods and investigate the asymptotic efficiency of the proposed approach. This work so expands the application scope of secondorder algorithms in stochastic optimization. |
| title | Online estimation of the inverse of the Hessian for stochastic optimization with application to universal stochastic Newton algorithms |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2401.10923 |