A Comparative Analysis of Optimization Methods for Classification on Various Datasets
Fuente:
Zenodo
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Recurso digital |
| Langue: | anglais |
| Publié: |
Zenodo
2025
|
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866901052188524544 |
|---|---|
| author | Simanta Das Soumitra Das |
| author_facet | Simanta Das Soumitra Das |
| contents | <p>Optimization, which involves the study of the conditions by which a variety of mathematical structures can be analyzed through the minimization or maximization of a function, is often seen as the heart of mathematics. In deep learning (DL), the scope of optimization broadly includes hyperparameter tuning, weight and bias adjustments, etc., until convergence of the loss or cost function (J), aiming to improve the model’s performance, prediction accuracy, and reliability in tasks like classification and regression. In recent years, the stochastic gradient algorithm and its variants are becoming widely used, and each offering various levels of success. The variants are called Adaptive Gradient Methods. A thorough comparison of adaptive gradient methods with respect to their convergence speed and Cross-Entropy Loss (CEL) in the mentioned classification tasks is provided; hence, the study covered optimization algorithms like SGD, Momentum SGD, RMSProp, Adam, Adagrad, Adadelta, Adamax, Nadam, and AMSGrad across three CNN architectures on MNIST, Fashion-MNIST, and CIFAR-10 datasets for 30 epochs. The optimizers that overall proved to be the best were SGD, RMSProp, Adam, and Nadam, while Adagrad and Adadelta showed consistent underperformance.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_15274192 |
| institution | Zenodo |
| language | eng |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | A Comparative Analysis of Optimization Methods for Classification on Various Datasets Simanta Das Soumitra Das <p>Optimization, which involves the study of the conditions by which a variety of mathematical structures can be analyzed through the minimization or maximization of a function, is often seen as the heart of mathematics. In deep learning (DL), the scope of optimization broadly includes hyperparameter tuning, weight and bias adjustments, etc., until convergence of the loss or cost function (J), aiming to improve the model’s performance, prediction accuracy, and reliability in tasks like classification and regression. In recent years, the stochastic gradient algorithm and its variants are becoming widely used, and each offering various levels of success. The variants are called Adaptive Gradient Methods. A thorough comparison of adaptive gradient methods with respect to their convergence speed and Cross-Entropy Loss (CEL) in the mentioned classification tasks is provided; hence, the study covered optimization algorithms like SGD, Momentum SGD, RMSProp, Adam, Adagrad, Adadelta, Adamax, Nadam, and AMSGrad across three CNN architectures on MNIST, Fashion-MNIST, and CIFAR-10 datasets for 30 epochs. The optimizers that overall proved to be the best were SGD, RMSProp, Adam, and Nadam, while Adagrad and Adadelta showed consistent underperformance.</p> |
| title | A Comparative Analysis of Optimization Methods for Classification on Various Datasets |
| url | https://doi.org/10.5281/zenodo.15274192 |