Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.18849 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908339800113152 |
|---|---|
| author | Naifar, Omar |
| author_facet | Naifar, Omar |
| contents | This paper introduces Tempered Fractional Gradient Descent (TFGD), a novel optimization framework that synergizes fractional calculus with exponential tempering to enhance gradient-based learning. Traditional gradient descent methods often suffer from oscillatory updates and slow convergence in high-dimensional, noisy landscapes. TFGD addresses these limitations by incorporating a tempered memory mechanism, where historical gradients are weighted by fractional coefficients $|w_j| = \binomα{j}$ and exponentially decayed via a tempering parameter $λ$. Theoretical analysis establishes TFGD's convergence guarantees: in convex settings, it achieves an $\mathcal{O}(1/K)$ rate with alignment coefficient $d_{α,λ} = (1 - e^{-λ})^{-α}$, while stochastic variants attain $\mathcal{O}(1/k^α)$ error decay. The algorithm maintains $\mathcal{O}(n)$ time complexity equivalent to SGD, with memory overhead scaling as $\mathcal{O}(d/λ)$ for parameter dimension $d$. Empirical validation on the Breast Cancer Wisconsin dataset demonstrates TFGD's superiority, achieving 98.25\% test accuracy (vs. 92.11\% for SGD) and 2$\times$ faster convergence. The tempered memory mechanism proves particularly effective in medical classification tasks, where feature correlations benefit from stable gradient averaging. These results position TFGD as a robust alternative to conventional optimizers in both theoretical and applied machine learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_18849 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Theoretical Framework for Tempered Fractional Gradient Descent: Application to Breast Cancer Classification Naifar, Omar Machine Learning Image and Video Processing 26A33, 90C26, 68T05, 92C50 G.1.6; I.2.6; J.3 This paper introduces Tempered Fractional Gradient Descent (TFGD), a novel optimization framework that synergizes fractional calculus with exponential tempering to enhance gradient-based learning. Traditional gradient descent methods often suffer from oscillatory updates and slow convergence in high-dimensional, noisy landscapes. TFGD addresses these limitations by incorporating a tempered memory mechanism, where historical gradients are weighted by fractional coefficients $|w_j| = \binomα{j}$ and exponentially decayed via a tempering parameter $λ$. Theoretical analysis establishes TFGD's convergence guarantees: in convex settings, it achieves an $\mathcal{O}(1/K)$ rate with alignment coefficient $d_{α,λ} = (1 - e^{-λ})^{-α}$, while stochastic variants attain $\mathcal{O}(1/k^α)$ error decay. The algorithm maintains $\mathcal{O}(n)$ time complexity equivalent to SGD, with memory overhead scaling as $\mathcal{O}(d/λ)$ for parameter dimension $d$. Empirical validation on the Breast Cancer Wisconsin dataset demonstrates TFGD's superiority, achieving 98.25\% test accuracy (vs. 92.11\% for SGD) and 2$\times$ faster convergence. The tempered memory mechanism proves particularly effective in medical classification tasks, where feature correlations benefit from stable gradient averaging. These results position TFGD as a robust alternative to conventional optimizers in both theoretical and applied machine learning. |
| title | Theoretical Framework for Tempered Fractional Gradient Descent: Application to Breast Cancer Classification |
| topic | Machine Learning Image and Video Processing 26A33, 90C26, 68T05, 92C50 G.1.6; I.2.6; J.3 |
| url | https://arxiv.org/abs/2504.18849 |