Copertina

Where Do Large Learning Rates Lead Us?

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Sadrtdinov, Ildus, Kodryan, Maxim, Pokonechny, Eduard, Lobacheva, Ekaterina, Vetrov, Dmitry
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

To Stay or Not to Stay in the Pre-train Basin: Insights on Ensembling in Transfer Learning
di: Sadrtdinov, Ildus, et al.
Pubblicazione: (2023)

SGD as Free Energy Minimization: A Thermodynamic View on Neural Network Training
di: Sadrtdinov, Ildus, et al.
Pubblicazione: (2025)

Can Stationary Distributions of Scale-Invariant Neural Networks Be Described by the Thermodynamics of an Ideal Gas?
di: Sadrtdinov, Ildus, et al.
Pubblicazione: (2025)

Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?
di: Shabalin, Alexander, et al.
Pubblicazione: (2026)

Unsupervised Process Reward Models
di: Gadetsky, Artyom, et al.
Pubblicazione: (2026)

Non-Euclidean SGD for Structured Optimization: Unified Analysis and Improved Rates
di: Kovalev, Dmitry, et al.
Pubblicazione: (2025)

SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations
di: Bartosh, Grigory, et al.
Pubblicazione: (2025)

Neural Diffusion Models
di: Bartosh, Grigory, et al.
Pubblicazione: (2023)

Generative Flow Networks as Entropy-Regularized RL
di: Tiapkin, Daniil, et al.
Pubblicazione: (2023)

Regularized Distribution Matching Distillation for One-step Unpaired Image-to-Image Translation
di: Rakitin, Denis, et al.
Pubblicazione: (2024)

Training Dynamics Underlying Language Model Scaling Laws: Loss Deceleration and Zero-Sum Learning
di: Mircea, Andrei, et al.
Pubblicazione: (2025)

Neural Flow Diffusion Models: Learnable Forward Process for Improved Diffusion Modelling
di: Bartosh, Grigory, et al.
Pubblicazione: (2024)

Streaming Generation of Co-Speech Gestures via Accelerated Rolling Diffusion
di: Vu, Evgeniia, et al.
Pubblicazione: (2025)

Do Large Language Models Reason Causally Like Us? Even Better?
di: Dettki, Hanna M., et al.
Pubblicazione: (2025)

Improving GFlowNets with Monte Carlo Tree Search
di: Morozov, Nikita, et al.
Pubblicazione: (2024)

Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layers
di: Grishina, Ekaterina, et al.
Pubblicazione: (2024)

On Linear Convergence in Smooth Convex-Concave Bilinearly-Coupled Saddle-Point Optimization: Lower Bounds and Optimal Algorithms
di: Kovalev, Dmitry, et al.
Pubblicazione: (2024)

Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimization
di: Borodich, Ekaterina, et al.
Pubblicazione: (2025)

ProcrustesGPT: Compressing LLMs with Structured Matrices and Orthogonal Transformations
di: Grishina, Ekaterina, et al.
Pubblicazione: (2025)

Generalization error of spectral algorithms
di: Velikanov, Maksim, et al.
Pubblicazione: (2024)

Adaptive Destruction Processes for Diffusion Samplers
di: Gritsaev, Timofei, et al.
Pubblicazione: (2025)

Guided Star-Shaped Masked Diffusion
di: Meshchaninov, Viacheslav, et al.
Pubblicazione: (2025)

GLGENN: A Novel Parameter-Light Equivariant Neural Networks Architecture Based on Clifford Geometric Algebras
di: Filimoshina, Ekaterina, et al.
Pubblicazione: (2025)

Where Do Reasoning Models Refuse?
di: Yamaguchi, Kureha, et al.
Pubblicazione: (2025)

Diffusion on language model encodings for protein sequence generation
di: Meshchaninov, Viacheslav, et al.
Pubblicazione: (2024)

Gradual Optimization Learning for Conformational Energy Minimization
di: Tsypin, Artem, et al.
Pubblicazione: (2023)

What Can Grokking Teach Us About Learning Under Nonstationarity?
di: Lyle, Clare, et al.
Pubblicazione: (2025)

How Do Transformers Learn to Associate Tokens: Gradient Leading Terms Bring Mechanistic Interpretability
di: Im, Shawn, et al.
Pubblicazione: (2026)

Lower Bounds and Optimal Algorithms for Non-Smooth Convex Decentralized Optimization over Time-Varying Networks
di: Kovalev, Dmitry, et al.
Pubblicazione: (2024)

Do LLMs Understand Romanian Driving Laws? A Study on Multimodal and Fine-Tuned Question Answering
di: Barbu, Eduard, et al.
Pubblicazione: (2025)

Where Do LLMs Still Struggle? An In-Depth Analysis of Code Generation Benchmarks
di: Sharifloo, Amir Molzam, et al.
Pubblicazione: (2025)

Where Do the Joules Go? Diagnosing Inference Energy Consumption
di: Chung, Jae-Won, et al.
Pubblicazione: (2026)

See Beyond a Single View: Multi-Attribution Learning Leads to Better Conversion Rate Prediction
di: Chen, Sishuo, et al.
Pubblicazione: (2025)

Where to Intervene: Action Selection in Deep Reinforcement Learning
di: Zhang, Wenbo, et al.
Pubblicazione: (2025)

Uncertainty Quantification for Data-Driven Machine Learning Models in Nuclear Engineering Applications: Where We Are and What Do We Need?
di: Wu, Xu, et al.
Pubblicazione: (2025)

Position: What Can Large Language Models Tell Us about Time Series Analysis
di: Jin, Ming, et al.
Pubblicazione: (2024)

Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates
di: Rammal, Ahmad, et al.
Pubblicazione: (2023)

Learning Where It Matters: Geometric Anchoring for Robust Preference Alignment
di: Cho, Youngjae, et al.
Pubblicazione: (2026)

Motivating Next-Gen Accelerators with Flexible (N:M) Activation Sparsity via Benchmarking Lightweight Post-Training Sparsification Approaches
di: Alanova, Shirin, et al.
Pubblicazione: (2025)

SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms
di: Abdukhakimov, Farshed, et al.
Pubblicazione: (2023)