Cover Image

Revealing Modular Gradient Noise Imbalance in LLMs: Calibrating Adam via Signal-to-Noise Ratio

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wen, Ziqing, Liu, Zhouyang, Wang, Jiahuan, Luo, Ping, Shen, Li, Li, Dongsheng, Sun, Tao
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FOAM: Blocked State Folding for Memory-Efficient LLM Training
by: Wen, Ziqing, et al.
Published: (2025)

GWT: Scalable Optimizer State Compression for Large Language Model Training
by: Wen, Ziqing, et al.
Published: (2025)

Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity
by: Luo, Ping, et al.
Published: (2026)

Resurrecting Label Propagation for Graphs with Heterophily and Label Noise
by: Cheng, Yao, et al.
Published: (2023)

Unveiling High-Probability Generalization in Decentralized SGD
by: Wang, Jiahuan, et al.
Published: (2026)

Stability and Generalization for Decentralized Markov SGD
by: Wang, Jiahuan, et al.
Published: (2026)

Noise Balance and Stationary Distribution of Stochastic Gradient Descent
by: Ziyin, Liu, et al.
Published: (2023)

NoiseAR: AutoRegressing Initial Noise Prior for Diffusion Models
by: Li, Zeming, et al.
Published: (2025)

Uncertainty-Aware Gradient Signal-to-Noise Data Selection for Instruction Tuning
by: Yuan, Zhihang, et al.
Published: (2026)

The Effect of Mini-Batch Noise on the Implicit Bias of Adam
by: Cattaneo, Matias D., et al.
Published: (2026)

Unveiling m-Sharpness Through the Structure of Stochastic Gradient Noise
by: Luo, Haocheng, et al.
Published: (2025)

CLEAR: Revealing How Noise and Ambiguity Degrade Reliability in LLMs for Medicine
by: Guo, Kevin H., et al.
Published: (2026)

Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
by: Luo, Ping, et al.
Published: (2024)

Behind the Noise: Conformal Quantile Regression Reveals Emergent Representations
by: Zwart, Petrus H., et al.
Published: (2025)

Digital Twin-Driven Zero-Shot Fault Diagnosis of Axial Piston Pumps Using Fluid-Borne Noise Signals
by: Dong, Chang, et al.
Published: (2025)

Imbalanced Gradients in RL Post-Training of Multi-Task LLMs
by: Wu, Runzhe, et al.
Published: (2025)

Towards Self-Robust LLMs: Intrinsic Prompt Noise Resistance via CoIPO
by: Yang, Xin, et al.
Published: (2026)

Rethinking Flexible Graph Similarity Computation: One-step Alignment with Global Guidance
by: Liu, Zhouyang, et al.
Published: (2025)

"Generative Models for Financial Time Series Data: Enhancing Signal-to-Noise Ratio and Addressing Data Scarcity in A-Share Market
by: Che, Guangming
Published: (2024)

Generalized Robot Learning Framework
by: Yan, Jiahuan, et al.
Published: (2024)

NMCSE: Noise-Robust Multi-Modal Coupling Signal Estimation Method via Optimal Transport for Cardiovascular Disease Detection
by: Zhang, Peihong, et al.
Published: (2025)

Lilith: Developmental Modular LLMs with Chemical Signaling
by: Farooqi, Mohid, et al.
Published: (2025)

GRM: Utility-Aware Jailbreak Attacks on Audio LLMs via Gradient-Ratio Masking
by: Wang, Yunqiang, et al.
Published: (2026)

Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models
by: Tice, Cameron, et al.
Published: (2024)

$\mathrm{D}^\mathrm{3}$-Predictor: Noise-Free Deterministic Diffusion for Dense Prediction
by: Xia, Changliang, et al.
Published: (2025)

Attack-Aware Noise Calibration for Differential Privacy
by: Kulynych, Bogdan, et al.
Published: (2024)

Enhancing Zero-Shot Time Series Forecasting in Off-the-Shelf LLMs via Noise Injection
by: Yin, Xingyou, et al.
Published: (2025)

Noise-corrected GRPO: From Noisy Rewards to Unbiased Gradients
by: Mansouri, Omar El, et al.
Published: (2025)

DN-CL: Deep Symbolic Regression against Noise via Contrastive Learning
by: Liu, Jingyi, et al.
Published: (2024)

Whispers in the Noise: Surrogate-Guided Concept Awakening via a Multi-Agent Framework
by: Sun, Mengyu, et al.
Published: (2026)

CCS: Controllable and Constrained Sampling with Diffusion Models via Initial Noise Perturbation
by: Song, Bowen, et al.
Published: (2025)

From Noise to Signal: When Outliers Seed New Topics
by: Zve, Evangelia, et al.
Published: (2026)

Noise-aware Client Selection for carbon-efficient Federated Learning via Gradient Norm Thresholding
by: Wilhelm, Patrick, et al.
Published: (2026)

A Survey of Graph Neural Networks in Real world: Imbalance, Noise, Privacy and OOD Challenges
by: Ju, Wei, et al.
Published: (2024)

NoiseBoost: Alleviating Hallucination with Noise Perturbation for Multimodal Large Language Models
by: Wu, Kai, et al.
Published: (2024)

NoiseRater: Meta-Learned Noise Valuation for Diffusion Model Training
by: Wu, Fang, et al.
Published: (2026)

NoisyICL: A Little Noise in Model Parameters Calibrates In-context Learning
by: Zhao, Yufeng, et al.
Published: (2024)

Policy Filtration for RLHF to Mitigate Noise in Reward Models
by: Zhang, Chuheng, et al.
Published: (2024)

Reliable Few-shot Learning under Dual Noises
by: Zhang, Ji, et al.
Published: (2025)

Revealing Temporal Label Noise in Multimodal Hateful Video Classification
by: Yang, Shuonan, et al.
Published: (2025)