Copertina

Stepping on the Edge: Curvature Aware Learning Rate Tuners

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Roulet, Vincent, Agarwala, Atish, Grill, Jean-Bastien, Swirszcz, Grzegorz, Blondel, Mathieu, Pedregosa, Fabian
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

On the Interplay Between Stepsize Tuning and Progressive Sharpening
di: Roulet, Vincent, et al.
Pubblicazione: (2023)

Per-example gradients: a new frontier for understanding and improving optimizers
di: Roulet, Vincent, et al.
Pubblicazione: (2025)

How far away are truly hyperparameter-free learning algorithms?
di: Kasimbeg, Priya, et al.
Pubblicazione: (2025)

The Elements of Differentiable Programming
di: Blondel, Mathieu, et al.
Pubblicazione: (2024)

High dimensional theory of two-phase optimizers
di: Agarwala, Atish
Pubblicazione: (2026)

Joint Learning of Energy-based Models and their Partition Function
di: Sander, Michael E., et al.
Pubblicazione: (2025)

Loss Functions and Operators Generated by f-Divergences
di: Roulet, Vincent, et al.
Pubblicazione: (2025)

Autoregressive Language Models are Secretly Energy-Based Models: Insights into the Lookahead Capabilities of Next-Token Prediction
di: Blondel, Mathieu, et al.
Pubblicazione: (2025)

High dimensional analysis reveals conservative sharpening and a stochastic edge of stability
di: Agarwala, Atish, et al.
Pubblicazione: (2024)

Cutting Some Slack for SGD with Adaptive Polyak Stepsizes
di: Gower, Robert M., et al.
Pubblicazione: (2022)

Feature learning as alignment: a structural property of gradient descent in non-linear neural networks
di: Beaglehole, Daniel, et al.
Pubblicazione: (2024)

Neglected Hessian component explains mysteries in Sharpness regularization
di: Dauphin, Yann N., et al.
Pubblicazione: (2024)

Black-box optimization of noisy functions with unknown smoothness
di: Grill, Jean-Bastien, et al.
Pubblicazione: (2026)

Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
di: Grill, Jean-Bastien, et al.
Pubblicazione: (2026)

To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions
di: Marshall, Noah, et al.
Pubblicazione: (2024)

Exact Risk Curves of signSGD in High-Dimensions: Quantifying Preconditioning and Noise-Compression Effects
di: Xiao, Ke Liang, et al.
Pubblicazione: (2024)

What do near-optimal learning rate schedules look like?
di: Naganuma, Hiroki, et al.
Pubblicazione: (2026)

Avoiding spurious sharpness minimization broadens applicability of SAM
di: Singh, Sidak Pal, et al.
Pubblicazione: (2025)

Learning with Fitzpatrick Losses
di: Rakotomandimby, Seta, et al.
Pubblicazione: (2024)

Stability-Aware Training of Machine Learning Force Fields with Differentiable Boltzmann Estimators
di: Raja, Sanjeev, et al.
Pubblicazione: (2024)

Learning with Local Search MCMC Layers
di: Vivier-Ardisson, Germain, et al.
Pubblicazione: (2025)

Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks
di: Qiu, Shikai, et al.
Pubblicazione: (2025)

Phases of Muon: When Muon Eclipses SignSGD
di: Paquette, Elliot, et al.
Pubblicazione: (2026)

Planning in entropy-regularized Markov decision processes and games
di: Grill, Jean-Bastien, et al.
Pubblicazione: (2026)

When is Momentum Extragradient Optimal? A Polynomial-Based Analysis
di: Kim, Junhyung Lyle, et al.
Pubblicazione: (2022)

How do Transformers perform In-Context Autoregressive Learning?
di: Sander, Michael E., et al.
Pubblicazione: (2024)

Unlearning in- vs. out-of-distribution data in LLMs under gradient-based method
di: Baluta, Teodora, et al.
Pubblicazione: (2024)

Sign-Symmetry Learning Rules are Robust Fine-Tuners
di: Berriche, Aymene, et al.
Pubblicazione: (2025)

Regularized Large Neighborhood Search
di: Vivier-Ardisson, Germain, et al.
Pubblicazione: (2026)

Differentiable Knapsack and Top-k Operators via Dynamic Programming
di: Vivier-Ardisson, Germain, et al.
Pubblicazione: (2026)

Performance-driven Constrained Optimal Auto-Tuner for MPC
di: Puigjaner, Albert Gassol, et al.
Pubblicazione: (2025)

Step-by-Step Guidance to Differential Anemia Diagnosis with Real-World Data and Deep Reinforcement Learning
di: Muyama, Lillian, et al.
Pubblicazione: (2024)

CAT: Curvature-Adaptive Transformers for Geometry-Aware Learning
di: Lin, Ryan Y., et al.
Pubblicazione: (2025)

ML$^2$Tuner: Efficient Code Tuning via Multi-Level Machine Learning Models
di: Cha, JooHyoung, et al.
Pubblicazione: (2024)

The Effectiveness of Curvature-Based Rewiring and the Role of Hyperparameters in GNNs Revisited
di: Tori, Floriano, et al.
Pubblicazione: (2024)

DemoTuner: Automatic Performance Tuning for Database Management Systems Based on Demonstration Reinforcement Learning
di: Dou, Hui, et al.
Pubblicazione: (2025)

A Selective Quantization Tuner for ONNX Models
di: Louloudakis, Nikolaos, et al.
Pubblicazione: (2025)

Routers in Vision Mixture of Experts: An Empirical Study
di: Liu, Tianlin, et al.
Pubblicazione: (2024)

AGFT: An Adaptive GPU Frequency Tuner for Real-Time LLM Inference Optimization
di: Ye, Zicong, et al.
Pubblicazione: (2025)

Gradient-free Continual Learning
di: Rypeść, Grzegorz
Pubblicazione: (2025)