Gradient Descent Algorithm Survey
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918218211262464 |
|---|---|
| author | Fucheng, Deng Wanjie, Wang Ao, Gong Xiaoqi, Wang Fan, Wang |
| author_facet | Fucheng, Deng Wanjie, Wang Ao, Gong Xiaoqi, Wang Fan, Wang |
| contents | Focusing on the practical configuration needs of optimization algorithms in deep learning, this article concentrates on five major algorithms: SGD, Mini-batch SGD, Momentum, Adam, and Lion. It systematically analyzes the core advantages, limitations, and key practical recommendations of each algorithm. The research aims to gain an in-depth understanding of these algorithms and provide a standardized reference for the reasonable selection, parameter tuning, and performance improvement of optimization algorithms in both academic research and engineering practice, helping to solve optimization challenges in different scales of models and various training scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_20725 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Gradient Descent Algorithm Survey Fucheng, Deng Wanjie, Wang Ao, Gong Xiaoqi, Wang Fan, Wang Machine Learning Artificial Intelligence Focusing on the practical configuration needs of optimization algorithms in deep learning, this article concentrates on five major algorithms: SGD, Mini-batch SGD, Momentum, Adam, and Lion. It systematically analyzes the core advantages, limitations, and key practical recommendations of each algorithm. The research aims to gain an in-depth understanding of these algorithms and provide a standardized reference for the reasonable selection, parameter tuning, and performance improvement of optimization algorithms in both academic research and engineering practice, helping to solve optimization challenges in different scales of models and various training scenarios. |
| title | Gradient Descent Algorithm Survey |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2511.20725 |