A Theoretical and Experimental Study of a Novel Adaptive Learning Algorithm
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913169625055232 |
|---|---|
| author | Kumari, Sakshi M, Shyam Kumar P, Sushmitha |
| author_facet | Kumari, Sakshi M, Shyam Kumar P, Sushmitha |
| contents | A crucial component of machine learning algorithms is minimizing loss functions with less computational cost and less oscillations. While adaptive learning rate-based optimizers have been widely used for real-world tasks, they do not guarantee convergence, which is why AMSGrad was later introduced to investigate the non-convergence behaviour of Adam. In this paper, popular adaptive optimization methods like Adam and AMSGrad are critically reviewed with an emphasis on their fundamental design concepts. To address limitations of the above mentioned optimizers, a new optimizer variant, C-Adam, is proposed based on the line of sight approach. A theoretical proof for convergence is also provided and the optimizer is validated through a number of real-life based numerical experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_29273 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Theoretical and Experimental Study of a Novel Adaptive Learning Algorithm Kumari, Sakshi M, Shyam Kumar P, Sushmitha Machine Learning Optimization and Control A crucial component of machine learning algorithms is minimizing loss functions with less computational cost and less oscillations. While adaptive learning rate-based optimizers have been widely used for real-world tasks, they do not guarantee convergence, which is why AMSGrad was later introduced to investigate the non-convergence behaviour of Adam. In this paper, popular adaptive optimization methods like Adam and AMSGrad are critically reviewed with an emphasis on their fundamental design concepts. To address limitations of the above mentioned optimizers, a new optimizer variant, C-Adam, is proposed based on the line of sight approach. A theoretical proof for convergence is also provided and the optimizer is validated through a number of real-life based numerical experiments. |
| title | A Theoretical and Experimental Study of a Novel Adaptive Learning Algorithm |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2605.29273 |