Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.18841 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917622391504896 |
|---|---|
| author | Li, Shuyao Wright, Stephen J. |
| author_facet | Li, Shuyao Wright, Stephen J. |
| contents | We consider minimization of a smooth nonconvex function with inexact oracle access to gradient and Hessian (without assuming access to the function value) to achieve approximate second-order optimality. A novel feature of our method is that if an approximate direction of negative curvature is chosen as the step, we choose its sense to be positive or negative with equal probability. We allow gradients to be inexact in a relative sense and relax the coupling between inexactness thresholds for the first- and second-order optimality conditions. Our convergence analysis includes both an expectation bound based on martingale analysis and a high-probability bound based on concentration inequalities. We apply our algorithm to empirical risk minimization problems and obtain improved gradient sample complexity over existing works. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_18841 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | A randomized algorithm for nonconvex minimization with inexact evaluations and complexity guarantees Li, Shuyao Wright, Stephen J. Optimization and Control Machine Learning We consider minimization of a smooth nonconvex function with inexact oracle access to gradient and Hessian (without assuming access to the function value) to achieve approximate second-order optimality. A novel feature of our method is that if an approximate direction of negative curvature is chosen as the step, we choose its sense to be positive or negative with equal probability. We allow gradients to be inexact in a relative sense and relax the coupling between inexactness thresholds for the first- and second-order optimality conditions. Our convergence analysis includes both an expectation bound based on martingale analysis and a high-probability bound based on concentration inequalities. We apply our algorithm to empirical risk minimization problems and obtain improved gradient sample complexity over existing works. |
| title | A randomized algorithm for nonconvex minimization with inexact evaluations and complexity guarantees |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2310.18841 |