Image de couverture de livre

The Lifecycle of the Spectral Edge: From Gradient Learning to Weight-Decay Compression

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Xu, Yongzhong
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Spectral Edge Dynamics Reveal Functional Modes of Learning
par: Xu, Yongzhong
Publié: (2026)

The Geometry of Multi-Task Grokking: Transverse Instability, Superposition, and Weight Decay Phase Structure
par: Xu, Yongzhong
Publié: (2026)

Spectral Edge Dynamics of Training Trajectories: Signal--Noise Geometry Across Scales
par: Xu, Yongzhong
Publié: (2026)

Spectral Edge Dynamics: An Analytical-Empirical Study of Phase Transitions in Neural Network Training
par: Xu, Yongzhong
Publié: (2026)

Feature Repulsion and Spectral Lock-in: An Empirical Study of Two-Layer Network Grokking
par: Xu, Yongzhong
Publié: (2026)

Gradient-Direction Sensitivity Reveals Linear-Centroid Coupling Hidden by Optimizer Trajectories
par: Xu, Yongzhong
Publié: (2026)

Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers
par: Xu, Yongzhong
Publié: (2026)

The Spectral Lifecycle of Transformer Training: Transient Compression Waves, Persistent Spectral Gradients, and the Q/K--V Asymmetry
par: Liu, Yi
Publié: (2026)

Global Low-Rank, Local Full-Rank: The Holographic Encoding of Learned Algorithms
par: Xu, Yongzhong
Publié: (2026)

Low-Dimensional Execution Manifolds in Transformer Learning Dynamics: Evidence from Modular Arithmetic Tasks
par: Xu, Yongzhong
Publié: (2026)

Early-Warning Signals of Grokking via Loss-Landscape Geometry
par: Xu, Yongzhong
Publié: (2026)

Low-Dimensional and Transversely Curved Optimization Dynamics in Grokking
par: Xu, Yongzhong
Publié: (2026)

When Do Attention Circuits Form? Developmental Trajectories of Capability and Attention-Sink Emergence Across Three 1B-ClassArchitectures
par: Xu, Yongzhong
Publié: (2026)

Optimizer-Induced Low-Dimensional Drift and Transverse Dynamics in Transformer Training
par: Xu, Yongzhong
Publié: (2026)

The Rank and Gradient Lost in Non-stationarity: Sample Weight Decay for Mitigating Plasticity Loss in Reinforcement Learning
par: Wu, Zihao, et autres
Publié: (2026)

On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective
par: Xie, Zeke, et autres
Publié: (2020)

Correction of Decoupled Weight Decay
par: Chou, Jason Chuan-Chih
Publié: (2025)

From $\log π$ to $π$: Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight
par: Fu, Xiaoliang, et autres
Publié: (2026)

Theoretical Analysis of Sparse Optimization with Reparameterization, Weight Decay, and Adaptive Learning Rate
par: Xu, Huangyu, et autres
Publié: (2026)

Cautious Weight Decay
par: Chen, Lizhang, et autres
Publié: (2025)

Learning to Forget: Continual Learning with Adaptive Weight Decay
par: Ramesh, Aditya A., et autres
Publié: (2026)

Why Do We Need Weight Decay in Modern Deep Learning?
par: D'Angelo, Francesco, et autres
Publié: (2023)

Uncertainty Weighted Gradients for Model Calibration
par: Lin, Jinxu, et autres
Publié: (2025)

Predicting LLM Compression Degradation from Spectral Statistics
par: Xu, Mingxue
Publié: (2026)

Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?
par: Zhang, Kaiqi, et autres
Publié: (2022)

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks
par: Kosson, Atli, et autres
Publié: (2023)

EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices
par: Sanyal, Arnab, et autres
Publié: (2025)

Weighted Temporal Decay Loss for Learning Wearable PPG Data with Sparse Clinical Labels
par: Chung, Yunsung, et autres
Publié: (2026)

A Self-Attentive Meta-Optimizer with Group-Adaptive Learning Rates and Weight Decay
par: Zhao, JiangBo, et autres
Publié: (2026)

Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay
par: Laus, Hannah, et autres
Publié: (2025)

Task Weighting through Gradient Projection for Multitask Learning
par: Bohn, Christian, et autres
Publié: (2024)

DP-AdamW: Investigating Decoupled Weight Decay and Bias Correction in Private Deep Learning
par: Chooi, Jay, et autres
Publié: (2025)

Weight Decay may matter more than muP for Learning Rate Transfer in Practice
par: Kosson, Atli, et autres
Publié: (2025)

Does Weight Decay Enhance Training Stability?
par: Saether, Marius, et autres
Publié: (2026)

Dynamic Skill Lifecycle Management for Agentic Reinforcement Learning
par: Shen, Junhao, et autres
Publié: (2026)

Towards Understanding Neural Collapse: The Effects of Batch Normalization and Weight Decay
par: Pan, Leyan, et autres
Publié: (2023)

SGD and Weight Decay Secretly Minimize the Rank of Your Neural Network
par: Galanti, Tomer, et autres
Publié: (2022)

Neural Weight Compression for Language Models
par: Ryu, Jegwang, et autres
Publié: (2025)

Revisiting Gradient Descent: A Dual-Weight Method for Improved Learning
par: Wang, Xi
Publié: (2025)

Explaining GNN Explanations with Edge Gradients
par: He, Jesse, et autres
Publié: (2025)