Cover Image

Stepsize anything: A unified learning rate schedule for budgeted-iteration training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tang, Anda, Dong, Yiming, Zeng, Yutao, Xun, zhou, Lin, Zhouchen
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the $O(\frac{\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\ell_1$ Norm
by: Li, Huan, et al.
Published: (2025)

Convergence Rate Analysis of LION
by: Dong, Yiming, et al.
Published: (2024)

Convergence Rate Analysis of the AdamW-Style Shampoo: Unifying One-Sided and Two-Sided Preconditioning
by: Li, Huan, et al.
Published: (2026)

HyperbolicLR: Epoch insensitive learning rate scheduler
by: Kim, Tae-Geun
Published: (2024)

What do near-optimal learning rate schedules look like?
by: Naganuma, Hiroki, et al.
Published: (2026)

Conda: Column-Normalized Adam for Training Large Language Models Faster
by: Wang, Junjie, et al.
Published: (2025)

Stochastic Approximation with Block Coordinate Optimal Stepsizes
by: Jiang, Tao, et al.
Published: (2025)

Constant Stepsize Q-learning: Distributional Convergence, Bias and Extrapolation
by: Zhang, Yixuan, et al.
Published: (2024)

Accelerated Gradient Tracking over Time-varying Graphs for Decentralized Optimization
by: Li, Huan, et al.
Published: (2021)

RL, but don't do anything I wouldn't do
by: Cohen, Michael K., et al.
Published: (2024)

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
by: Hu, Lexiang, et al.
Published: (2025)

Governing Equation Discovery from Data Based on Differential Invariants
by: Hu, Lexiang, et al.
Published: (2025)

Symmetry Discovery for Different Data Types
by: Hu, Lexiang, et al.
Published: (2024)

Pareto Adversarial Robustness: Balancing Spatial Robustness and Sensitivity-based Robustness
by: Sun, Ke, et al.
Published: (2021)

Incorporating Arbitrary Matrix Group Equivariance into KANs
by: Hu, Lexiang, et al.
Published: (2024)

Improving Model Representation and Reducing KV Cache via Skip Connections with First Value Heads
by: Wu, Zhoutong, et al.
Published: (2025)

Universal pre-training by iterated random computation
by: Bloem, Peter
Published: (2025)

pUniFind: a unified large pre-trained deep learning model pushing the limit of mass spectra interpretation
by: Zhao, Jiale, et al.
Published: (2025)

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes
by: Orvieto, Antonio, et al.
Published: (2024)

Generalist++: A Meta-learning Framework for Mitigating Trade-off in Adversarial Training
by: Wang, Yisen, et al.
Published: (2025)

A training regime to learn unified representations from complementary breast imaging modalities
by: Sharma, Umang, et al.
Published: (2024)

Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes
by: Zhang, Ruiqi, et al.
Published: (2025)

On the Limitations and Capabilities of Position Embeddings for Length Generalization
by: Chen, Yang, et al.
Published: (2025)

Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers
by: Chen, Yihong, et al.
Published: (2026)

Beyond likelihood ratio bias: Nested multi-time-scale stochastic approximation for likelihood-free parameter estimation
by: Li, Zehao, et al.
Published: (2024)

Decision-focused learning for optimal PV-Battery scheduling
by: Depoortere, Joris, et al.
Published: (2026)

Variational operator learning: A unified paradigm marrying training neural operators and solving partial differential equations
by: Xu, Tengfei, et al.
Published: (2023)

Coercing LLMs to do and reveal (almost) anything
by: Geiping, Jonas, et al.
Published: (2024)

Rademacher learning rates for iterated random functions
by: Sandrić, Nikola
Published: (2025)

Low-Dimension-to-High-Dimension Generalization And Its Implications for Length Generalization
by: Chen, Yang, et al.
Published: (2024)

Partial Feedback Online Learning
by: Shao, Shihao, et al.
Published: (2026)

Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression
by: Wu, Jingfeng, et al.
Published: (2025)

Constant Stepsize Local GD for Logistic Regression: Acceleration by Instability
by: Crawshaw, Michael, et al.
Published: (2025)

On Suppressing Range of Adaptive Stepsizes of Adam to Improve Generalisation Performance
by: Zhang, Guoqiang
Published: (2023)

Posterior Approximation using Stochastic Gradient Ascent with Adaptive Stepsize
by: Lim, Kart-Leong, et al.
Published: (2024)

DIGIC: Domain Generalizable Imitation Learning by Causal Discovery
by: Chen, Yang, et al.
Published: (2024)

Relational Learning in Pre-Trained Models: A Theory from Hypergraph Recovery Perspective
by: Chen, Yang, et al.
Published: (2024)

Adaptive Stepsizing for Stochastic Gradient Langevin Dynamics in Bayesian Neural Networks
by: Rajpal, Rajit, et al.
Published: (2025)

Offline reinforcement learning for job-shop scheduling problems
by: Echeverria, Imanol, et al.
Published: (2024)

Regret of exploratory policy improvement and $q$-learning
by: Tang, Wenpin, et al.
Published: (2024)