Adjusted Shuffling SARAH: Advancing Complexity Analysis via Dynamic Gradient Weighting
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917536965066752 |
|---|---|
| author | Nguyen, Duc Toan Tran, Trang H. Nguyen, Lam M. |
| author_facet | Nguyen, Duc Toan Tran, Trang H. Nguyen, Lam M. |
| contents | In this paper, we propose Adjusted Shuffling SARAH, a novel algorithm that integrates shuffling strategies into the recursive SARAH framework using a dynamic weighting mechanism to enhance exploration. We analyze the algorithm under two operating modes. First, we show that the Exact Mode matches the best-known theoretical guarantees for shuffling variance-reduced methods in both strongly convex and non-convex settings. Second, to address large-scale regimes, we introduce an Inexact Mode that utilizes mini-batch estimators. A key contribution of our work is proving that this Inexact Mode achieves a total complexity independent of the dataset size, making it significantly more scalable than existing shuffling methods when the sample size is large. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_12444 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Adjusted Shuffling SARAH: Advancing Complexity Analysis via Dynamic Gradient Weighting Nguyen, Duc Toan Tran, Trang H. Nguyen, Lam M. Optimization and Control Machine Learning In this paper, we propose Adjusted Shuffling SARAH, a novel algorithm that integrates shuffling strategies into the recursive SARAH framework using a dynamic weighting mechanism to enhance exploration. We analyze the algorithm under two operating modes. First, we show that the Exact Mode matches the best-known theoretical guarantees for shuffling variance-reduced methods in both strongly convex and non-convex settings. Second, to address large-scale regimes, we introduce an Inexact Mode that utilizes mini-batch estimators. A key contribution of our work is proving that this Inexact Mode achieves a total complexity independent of the dataset size, making it significantly more scalable than existing shuffling methods when the sample size is large. |
| title | Adjusted Shuffling SARAH: Advancing Complexity Analysis via Dynamic Gradient Weighting |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2506.12444 |