:: Library Catalog

Image de couverture de livre

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Dremov, Aleksandr, Hägele, Alexander, Kosson, Atli, Jaggi, Martin
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	https://arxiv.org/abs/2508.01483
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training
par: Kosson, Atli, et autres
Publié: (2024)

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks
par: Kosson, Atli, et autres
Publié: (2023)

Weight Decay may matter more than muP for Learning Rate Transfer in Practice
par: Kosson, Atli, et autres
Publié: (2025)

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations
par: Hägele, Alexander, et autres
Publié: (2024)

Compute-Optimal Quantization-Aware Training
par: Dremov, Aleksandr, et autres
Publié: (2025)

Optimal Learning-Rate Schedules under Functional Scaling Laws: Power Decay and Warmup-Stable-Decay
par: Li, Binghui, et autres
Publié: (2026)

Efficient Generative Model Training via Embedded Representation Warmup
par: Liu, Deyuan, et autres
Publié: (2025)

Data Warmup: Complexity-Aware Curricula for Efficient Diffusion Training
par: Lin, Jinhong, et autres
Publié: (2026)

Optimal Linear Decay Learning Rate Schedules and Further Refinements
par: Defazio, Aaron, et autres
Publié: (2023)

Test-Time Warmup for Multimodal Large Language Models
par: Rajaneesh, Nikita, et autres
Publié: (2025)

Hierarchical Mixture-of-Experts with Two-Stage Optimization
par: Molodtsov, Gleb, et autres
Publié: (2026)

Universal Dynamics of Warmup Stable Decay: understanding WSD beyond Transformers
par: Belloni, Annalisa, et autres
Publié: (2026)

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling
par: Meterez, Alexandru, et autres
Publié: (2025)

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective
par: Wen, Kaiyue, et autres
Publié: (2024)

ScheduleFree+: Scaling Learning-Rate-Free & Schedule-Free Learning to Large Language Models
par: Defazio, Aaron
Publié: (2026)

Dynamic Low-rank Approximation of Full-Matrix Preconditioner for Training Generalized Linear Models
par: Matveeva, Tatyana, et autres
Publié: (2025)

Cyclical Log Annealing as a Learning Rate Scheduler
par: Naveen, Philip
Publié: (2024)

The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training
par: Schaipp, Fabian, et autres
Publié: (2025)

NeuralGrok: Accelerate Grokking by Neural Gradient Transformation
par: Zhou, Xinyu, et autres
Publié: (2025)

Super Level Sets and Exponential Decay: A Synergistic Approach to Stable Neural Network Training
par: Chaudhary, Jatin, et autres
Publié: (2024)

Theoretical Analysis of Sparse Optimization with Reparameterization, Weight Decay, and Adaptive Learning Rate
par: Xu, Huangyu, et autres
Publié: (2026)

DoGE: Domain Reweighting with Generalization Estimation
par: Fan, Simin, et autres
Publié: (2023)

Towards an empirical understanding of MoE design choices
par: Fan, Dongyang, et autres
Publié: (2024)

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler
par: Shen, Yikang, et autres
Publié: (2024)

Heterogeneous Learning Rate Scheduling for Neural Architecture Search on Long-Tailed Datasets
par: Tang, Chenxia
Publié: (2024)

Two-Stage Representation Learning for Analyzing Movement Behavior Dynamics in People Living with Dementia
par: Cui, Jin, et autres
Publié: (2025)

Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs
par: Bossy, Thierry, et autres
Publié: (2025)

Load-Aware Training Scheduling for Model Circulation-based Decentralized Federated Learning
par: Kainuma, Haruki, et autres
Publié: (2025)

Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages
par: Ma, Guozheng, et autres
Publié: (2023)

$α$-LoRA: Effective Fine-Tuning via Base Model Rescaling
par: Firdoussi, Aymane El, et autres
Publié: (2025)

Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging
par: Meterez, Alexandru, et autres
Publié: (2026)

Beyond URLs: Metadata Diversity and Position for Efficient LLM Pretraining
par: Fan, Dongyang, et autres
Publié: (2025)

A theoretical framework for self-supervised contrastive learning for continuous dependent data
par: Marusov, Alexander, et autres
Publié: (2025)

Unsupervised Physics-Informed Operator Learning through Multi-Stage Curriculum Training
par: Marcandelli, Paolo, et autres
Publié: (2026)

How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining
par: Luo, Kairong, et autres
Publié: (2025)

Dynamic Temperature Scheduler for Knowledge Distillation
par: Islam, Sibgat Ul, et autres
Publié: (2025)

Analyzing Patient Daily Movement Behavior Dynamics Using Two-Stage Encoding Model
par: Cui, Jin, et autres
Publié: (2025)

Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning
par: Qiu, Haomiao, et autres
Publié: (2025)

A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules
par: Luo, Kairong, et autres
Publié: (2025)

Beyond Training: Optimizing Reinforcement Learning Based Job Shop Scheduling Through Adaptive Action Sampling
par: de Puiseau, Constantin Waubert, et autres
Publié: (2024)