Cover Image

SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Huang, Tianjin, Zhu, Ziquan, Jin, Gaojie, Liu, Lu, Wang, Zhangyang, Liu, Shiwei
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Margin-Adaptive Confidence Ranking for Reliable LLM Judgement
by: Jin, Gaojie, et al.
Published: (2026)

LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning
by: Liu, Zihang, et al.
Published: (2025)

GradientStabilizer:Fix the Norm, Not the Gradient
by: Huang, Tianjin, et al.
Published: (2025)

Knowledge Boundary Discovery for Large Language Models
by: Wang, Ziquan, et al.
Published: (2026)

Junk DNA Hypothesis: Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs
by: Yin, Lu, et al.
Published: (2023)

Outlier-weighed Layerwise Sampling for LLM Fine-tuning
by: Li, Pengxiang, et al.
Published: (2024)

POT: Inducing Overthinking in LLMs via Black-Box Iterative Optimization
by: Li, Xinyu, et al.
Published: (2025)

Enhancing Adversarial Training via Reweighting Optimization Trajectory
by: Huang, Tianjin, et al.
Published: (2023)

Muon is Scalable for LLM Training
by: Liu, Jingyuan, et al.
Published: (2025)

AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training
by: Zhang, Huishuai, et al.
Published: (2025)

BAGEN: Are LLM Agents Budget-Aware?
by: Lin, Yuxiang, et al.
Published: (2026)

Dataset Reset Policy Optimization for RLHF
by: Chang, Jonathan D., et al.
Published: (2024)

The Compositional Architecture of Regret in Large Language Models
by: Cui, Xiangxiang, et al.
Published: (2025)

Learn More, Forget Less: A Gradient-Aware Data Selection Approach for LLM
by: Liu, Yibai, et al.
Published: (2025)

BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
by: Huang, Wei, et al.
Published: (2024)

Direct Training Needs Regularisation: Anytime Optimal Inference Spiking Neural Network
by: Wu, Dengyu, et al.
Published: (2024)

Memory-Efficient LLM Training with Online Subspace Descent
by: Liang, Kaizhao, et al.
Published: (2024)

Training-Trajectory-Aware Token Selection
by: Shen, Zhanming, et al.
Published: (2026)

LoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuning
by: Perin, Gabriel J., et al.
Published: (2025)

Adaptive-Boundary-Clipping GRPO: Ensuring Bounded Ratios for Stable and Generalizable Training
by: Liu, Chi, et al.
Published: (2026)

SpikingBrain: Spiking Brain-inspired Large Models
by: Pan, Yuqi, et al.
Published: (2025)

Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training
by: Liu, Yixin, et al.
Published: (2026)

LoopTool: Closing the Data-Training Loop for Robust LLM Tool Calls
by: Zhang, Kangning, et al.
Published: (2025)

Momentum Streams for Optimizer-Inspired Transformers
by: Gai, Jingchu, et al.
Published: (2026)

EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
by: Pan, Chengjun, et al.
Published: (2026)

Time-Aware Feature Selection: Adaptive Temporal Masking for Stable Sparse Autoencoder Training
by: Li, T. Ed, et al.
Published: (2025)

POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformation
by: Qiu, Zeju, et al.
Published: (2026)

Agentic Critical Training
by: Liu, Weize, et al.
Published: (2026)

ExpeL: LLM Agents Are Experiential Learners
by: Zhao, Andrew, et al.
Published: (2023)

Efficient Agent Training for Computer Use
by: He, Yanheng, et al.
Published: (2025)

Better Embeddings with Coupled Adam
by: Stollenwerk, Felix, et al.
Published: (2025)

On Designing Effective RL Reward at Training Time for LLM Reasoning
by: Gao, Jiaxuan, et al.
Published: (2024)

ConsistRM: Improving Generative Reward Models via Consistency-Aware Self-Training
by: Liang, Yu, et al.
Published: (2026)

Reparameterized LLM Training via Orthogonal Equivalence Transformation
by: Qiu, Zeju, et al.
Published: (2025)

LLM-Barber: Block-Aware Rebuilder for Sparsity Mask in One-Shot for Large Language Models
by: Su, Yupeng, et al.
Published: (2024)

xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning
by: Qian, Cheng, et al.
Published: (2025)

Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective
by: Gan, Zeyu, et al.
Published: (2024)

$Q\sharp$: Provably Optimal Distributional RL for LLM Post-Training
by: Zhou, Jin Peng, et al.
Published: (2025)

MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning
by: Lu, Yiyang, et al.
Published: (2026)

CluCERT: Certifying LLM Robustness via Clustering-Guided Denoising Smoothing
by: Wang, Zixia, et al.
Published: (2025)