Adaptive Preconditioned Gradient Descent with Energy
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913390975254528 |
|---|---|
| author | Liu, Hailiang Nurbekyan, Levon Tian, Xuping Yang, Yunan |
| author_facet | Liu, Hailiang Nurbekyan, Levon Tian, Xuping Yang, Yunan |
| contents | We propose an adaptive step size with an energy approach for a suitable class of preconditioned gradient descent methods. We focus on settings where the preconditioning is applied to address the constraints in optimization problems, such as the Hessian-Riemannian and natural gradient descent methods. More specifically, we incorporate these preconditioned gradient descent algorithms in the recently introduced Adaptive Energy Gradient Descent (AEGD) framework. In particular, we discuss theoretical results on the unconditional energy-stability and convergence rates across three classes of objective functions. Furthermore, our numerical results demonstrate excellent performance of the proposed method on several test bed optimization problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_06733 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Adaptive Preconditioned Gradient Descent with Energy Liu, Hailiang Nurbekyan, Levon Tian, Xuping Yang, Yunan Optimization and Control Numerical Analysis 65K10, 90C26 We propose an adaptive step size with an energy approach for a suitable class of preconditioned gradient descent methods. We focus on settings where the preconditioning is applied to address the constraints in optimization problems, such as the Hessian-Riemannian and natural gradient descent methods. More specifically, we incorporate these preconditioned gradient descent algorithms in the recently introduced Adaptive Energy Gradient Descent (AEGD) framework. In particular, we discuss theoretical results on the unconditional energy-stability and convergence rates across three classes of objective functions. Furthermore, our numerical results demonstrate excellent performance of the proposed method on several test bed optimization problems. |
| title | Adaptive Preconditioned Gradient Descent with Energy |
| topic | Optimization and Control Numerical Analysis 65K10, 90C26 |
| url | https://arxiv.org/abs/2310.06733 |