Covariant Gradient Descent
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916685968048128 |
|---|---|
| author | Guskov, Dmitry Vanchurin, Vitaly |
| author_facet | Guskov, Dmitry Vanchurin, Vitaly |
| contents | We present a manifestly covariant formulation of the gradient descent method, ensuring consistency across arbitrary coordinate systems and general curved trainable spaces. The optimization dynamics is defined using a covariant force vector and a covariant metric tensor, both computed from the first and second statistical moments of the gradients. These moments are estimated through time-averaging with an exponential weight function, which preserves linear computational complexity. We show that commonly used optimization methods such as RMSProp, Adam and AdaBelief correspond to special limits of the covariant gradient descent (CGD) and demonstrate how these methods can be further generalized and improved. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_05279 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Covariant Gradient Descent Guskov, Dmitry Vanchurin, Vitaly Machine Learning High Energy Physics - Theory Optimization and Control We present a manifestly covariant formulation of the gradient descent method, ensuring consistency across arbitrary coordinate systems and general curved trainable spaces. The optimization dynamics is defined using a covariant force vector and a covariant metric tensor, both computed from the first and second statistical moments of the gradients. These moments are estimated through time-averaging with an exponential weight function, which preserves linear computational complexity. We show that commonly used optimization methods such as RMSProp, Adam and AdaBelief correspond to special limits of the covariant gradient descent (CGD) and demonstrate how these methods can be further generalized and improved. |
| title | Covariant Gradient Descent |
| topic | Machine Learning High Energy Physics - Theory Optimization and Control |
| url | https://arxiv.org/abs/2504.05279 |