Imagen de Portada

Why Gradients Rapidly Increase Near the End of Training

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Defazio, Aaron
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

ScheduleFree+: Scaling Learning-Rate-Free & Schedule-Free Learning to Large Language Models
por: Defazio, Aaron
Publicado: (2026)

Prodigy: An Expeditiously Adaptive Parameter-Free Learner
por: Mishchenko, Konstantin, et al.
Publicado: (2023)

Smoothing DiLoCo with Primal Averaging for Faster Training of LLMs
por: Defazio, Aaron, et al.
Publicado: (2025)

Optimal Linear Decay Learning Rate Schedules and Further Refinements
por: Defazio, Aaron, et al.
Publicado: (2023)

Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing its Preconditioner
por: Eschenhagen, Runa, et al.
Publicado: (2025)

Conditional Rectified Flow-based End-to-End Rapid Seismic Inversion Method
por: Xu, Haofei, et al.
Publicado: (2026)

The Road Less Scheduled
por: Defazio, Aaron, et al.
Publicado: (2024)

End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost
por: Tan, Qitao, et al.
Publicado: (2025)

Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning
por: Zhang, Kehao, et al.
Publicado: (2026)

Why Does Stochastic Gradient Descent Slow Down in Low-Precision Training?
por: Yun, Vincent-Daniel
Publicado: (2025)

Revisiting the Relationship between Adversarial and Clean Training: Why Clean Training Can Make Adversarial Training Better
por: Zhou, MingWei, et al.
Publicado: (2025)

L-MoE: End-to-End Training of a Lightweight Mixture of Low-Rank Adaptation Experts
por: Ji, Shihao, et al.
Publicado: (2025)

Why Inference in Large Models Becomes Decomposable After Training
por: Jin, Jidong
Publicado: (2026)

Gradient-Congruity Guided Federated Sparse Training
por: Tian, Chris Xing, et al.
Publicado: (2024)

Gradient-Free Training of Quantized Neural Networks
por: Cohen, Noa, et al.
Publicado: (2024)

Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention
por: Qiu, Haiquan, et al.
Publicado: (2025)

Gradient Multi-Normalization for Stateless and Scalable LLM Training
por: Scetbon, Meyer, et al.
Publicado: (2025)

Certification for Differentially Private Prediction in Gradient-Based Training
por: Wicker, Matthew, et al.
Publicado: (2024)

GIO: Gradient Information Optimization for Training Dataset Selection
por: Everaert, Dante, et al.
Publicado: (2023)

Why Adam Works Better with $β_1 = β_2$: The Missing Gradient Scale Invariance Principle
por: Fernández-Hernández, Alberto, et al.
Publicado: (2026)

Gradient Inversion Transcript: Leveraging Robust Generative Priors to Reconstruct Training Data from Gradient Leakage
por: Chen, Xinping, et al.
Publicado: (2025)

Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants
por: Huang, Vincent, et al.
Publicado: (2025)

Uncovering Gradient Inversion Risks in Practical Language Model Training
por: Feng, Xinguo, et al.
Publicado: (2025)

Imbalanced Gradients in RL Post-Training of Multi-Task LLMs
por: Wu, Runzhe, et al.
Publicado: (2025)

Post-Training with Policy Gradients: Optimality and the Base Model Barrier
por: Mousavi-Hosseini, Alireza, et al.
Publicado: (2026)

Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptation
por: Zhang, Xiyang, et al.
Publicado: (2026)

AnyBipe: An End-to-End Framework for Training and Deploying Bipedal Robots Guided by Large Language Models
por: Yao, Yifei, et al.
Publicado: (2024)

Accelerating RLHF Training with Reward Variance Increase
por: Yang, Zonglin, et al.
Publicado: (2025)

When and Why Adversarial Training Improves PINNs: A Neural Tangent Kernel Perspective
por: Cao, Yuan-dong, et al.
Publicado: (2026)

FullCert: Deterministic End-to-End Certification for Training and Inference of Neural Networks
por: Lorenz, Tobias, et al.
Publicado: (2024)

TSPulse: Tiny Pre-Trained Models with Disentangled Representations for Rapid Time-Series Analysis
por: Ekambaram, Vijay, et al.
Publicado: (2025)

Training Large Language Models to Reason via EM Policy Gradient
por: Xu, Tianbing
Publicado: (2025)

Understanding Gradient Boosting Classifier: Training, Prediction, and the Role of $γ_j$
por: Chen, Hung-Hsuan
Publicado: (2024)

Gradient Weight-normalized Low-rank Projection for Efficient LLM Training
por: Huang, Jia-Hong, et al.
Publicado: (2024)

Dynamic Tokenization via Reinforcement Patching: End-to-end Training and Zero-shot Transfer
por: Wu, Yulun, et al.
Publicado: (2026)

Training and Simulation of Quadrupedal Robot in Adaptive Stair Climbing for Indoor Firefighting: An End-to-End Reinforcement Learning Approach
por: Huang, Baixiao, et al.
Publicado: (2026)

RapidGNN: Energy and Communication-Efficient Distributed Training on Large-Scale Graph Neural Networks
por: Niam, Arefin, et al.
Publicado: (2025)

Gradient-Informed Temporal Sampling Improves Rollout Accuracy in PDE Surrogate Training
por: Wang, Wenshuo, et al.
Publicado: (2026)

GAC: Stabilizing Asynchronous RL Training for LLMs via Gradient Alignment Control
por: Xu, Haofeng, et al.
Publicado: (2026)

Beyond the Mean: Fisher-Orthogonal Projection for Natural Gradient Descent in Large Batch Training
por: Lu, Yishun, et al.
Publicado: (2025)