Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.16731 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908600256954368 |
|---|---|
| author | Shen, Ya Li, Qing-Na Dai, Yu-Hong |
| author_facet | Shen, Ya Li, Qing-Na Dai, Yu-Hong |
| contents | Gradient methods are among the simplest yet most widely used algorithms for unconstrained optimization. Motivated by a geometric property of the steepest descent (SD) method that can alleviate the zigzag behavior in quadratic problems, we develop a new gradient variant called the Triangle Steepest Descent (TSD) method. The TSD method introduces a cycle parameter $j$ that governs the periodic combination of past search directions, providing a geometry-driven mechanism to enhance convergence. To the best of our knowledge, TSD is the first formally established geometry-based gradient scheme since Akaike (1959). We prove that TSD is at least R-linearly convergent for strongly convex quadratic problems and demonstrate through extensive numerical experiments that it exhibits superlinear behavior, outperforming the Barzilai-Borwein (BB) method and monotone Dai-Yuan gradient method (DY) in quadratic cases. These results suggest that incorporating geometric information into gradient directions offers a promising avenue for developing efficient optimization algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_16731 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Triangle Steepest Descent: A Geometry-Based Gradient Algorithm with Guaranteed R-Linear Convergence Shen, Ya Li, Qing-Na Dai, Yu-Hong Optimization and Control 90C20 \and 90C25 Gradient methods are among the simplest yet most widely used algorithms for unconstrained optimization. Motivated by a geometric property of the steepest descent (SD) method that can alleviate the zigzag behavior in quadratic problems, we develop a new gradient variant called the Triangle Steepest Descent (TSD) method. The TSD method introduces a cycle parameter $j$ that governs the periodic combination of past search directions, providing a geometry-driven mechanism to enhance convergence. To the best of our knowledge, TSD is the first formally established geometry-based gradient scheme since Akaike (1959). We prove that TSD is at least R-linearly convergent for strongly convex quadratic problems and demonstrate through extensive numerical experiments that it exhibits superlinear behavior, outperforming the Barzilai-Borwein (BB) method and monotone Dai-Yuan gradient method (DY) in quadratic cases. These results suggest that incorporating geometric information into gradient directions offers a promising avenue for developing efficient optimization algorithms. |
| title | Triangle Steepest Descent: A Geometry-Based Gradient Algorithm with Guaranteed R-Linear Convergence |
| topic | Optimization and Control 90C20 \and 90C25 |
| url | https://arxiv.org/abs/2501.16731 |