Imagen de Portada

Gradient Multi-Normalization for Stateless and Scalable LLM Training

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Scetbon, Meyer, Ma, Chao, Gong, Wenbo, Meeds, Edward
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training
por: Ma, Chao, et al.
Publicado: (2024)

Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension
por: Gong, Wenbo, et al.
Publicado: (2025)

MASP: Scalable GNN-based Planning for Multi-Agent Navigation
por: Yang, Xinyi, et al.
Publicado: (2023)

The Essential Role of Causality in Foundation World Models for Embodied AI
por: Gupta, Tarun, et al.
Publicado: (2024)

Does Your Optimizer Care How You Normalize? Normalization-Optimizer Coupling in LLM Training
por: Abouzeid, Abdelrahman
Publicado: (2026)

Muon is Scalable for LLM Training
por: Liu, Jingyuan, et al.
Publicado: (2025)

Comprehensive Forecasting-Based Analysis of Hybrid and Stacked Stateful/ Stateless Models
por: Saha, Swayamjit
Publicado: (2024)

Scalable Multi-Objective and Meta Reinforcement Learning via Gradient Estimation
por: Zhang, Zhenshuo, et al.
Publicado: (2025)

Gradient Weight-normalized Low-rank Projection for Efficient LLM Training
por: Huang, Jia-Hong, et al.
Publicado: (2024)

Automating Deception: Scalable Multi-Turn LLM Jailbreaks
por: Kumarappan, Adarsh, et al.
Publicado: (2025)

GraSS: Scalable Data Attribution with Gradient Sparsification and Sparse Projection
por: Hu, Pingbang, et al.
Publicado: (2025)

Imbalanced Gradients in RL Post-Training of Multi-Task LLMs
por: Wu, Runzhe, et al.
Publicado: (2025)

Uncovering Gradient Inversion Risks in Practical Language Model Training
por: Feng, Xinguo, et al.
Publicado: (2025)

Axiomatization of Gradient Smoothing in Neural Networks
por: Zhou, Linjiang, et al.
Publicado: (2024)

LeanTTA: A Backpropagation-Free and Stateless Approach to Quantized Test-Time Adaptation on Edge Devices
por: Dong, Cynthia, et al.
Publicado: (2025)

Low-Rank Correction for Quantized LLMs
por: Scetbon, Meyer, et al.
Publicado: (2024)

Mitigating Gradient Overlap in Deep Residual Networks with Gradient Normalization for Improved Non-Convex Optimization
por: Yun, Juyoung
Publicado: (2024)

Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMs
por: Salem, Ahmed, et al.
Publicado: (2026)

EDGC: Entropy-driven Dynamic Gradient Compression for Efficient LLM Training
por: Yi, Qingao, et al.
Publicado: (2025)

FlowPG: Action-constrained Policy Gradient with Normalizing Flows
por: Brahmanage, Janaka Chathuranga, et al.
Publicado: (2024)

GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projection
por: Su, DiJia, et al.
Publicado: (2025)

Lotus: Efficient LLM Training by Randomized Low-Rank Gradient Projection with Adaptive Subspace Switching
por: Miao, Tianhao, et al.
Publicado: (2026)

MALT: Improving Reasoning with Multi-Agent LLM Training
por: Motwani, Sumeet Ramesh, et al.
Publicado: (2024)

Wasserstein Gradient Flows for Scalable and Regularized Barycenter Computation
por: Montesuma, Eduardo Fernandes, et al.
Publicado: (2025)

Neural Structure Learning with Stochastic Differential Equations
por: Wang, Benjie, et al.
Publicado: (2023)

Counter-Samples: A Stateless Strategy to Neutralize Black Box Adversarial Attacks
por: Bokobza, Roey, et al.
Publicado: (2024)

ARO: A New Lens On Matrix Optimization For Large Models
por: Gong, Wenbo, et al.
Publicado: (2026)

Descent-Guided Policy Gradient for Scalable Cooperative Multi-Agent Learning
por: Yang, Shan, et al.
Publicado: (2026)

Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training
por: Hu, Pingbang, et al.
Publicado: (2026)

Transformers are Stateless Differentiable Neural Computers
por: Tang, Bo, et al.
Publicado: (2026)

Rooted Absorbed Prefix Trajectory Balance with Submodular Replay for GFlowNet Training
por: Wang, Xi, et al.
Publicado: (2026)

LLM-ML Teaming: Integrated Symbolic Decoding and Gradient Search for Valid and Stable Generative Feature Transformation
por: Wang, Xinyuan, et al.
Publicado: (2025)

FwdLLM: Efficient FedLLM using Forward Gradient
por: Xu, Mengwei, et al.
Publicado: (2023)

Efficient Regression-Based Training of Normalizing Flows for Boltzmann Generators
por: Rehman, Danyal, et al.
Publicado: (2025)

Gradient-Congruity Guided Federated Sparse Training
por: Tian, Chris Xing, et al.
Publicado: (2024)

Gradient-Free Training of Quantized Neural Networks
por: Cohen, Noa, et al.
Publicado: (2024)

E^2-LLM: Bridging Neural Signals and Interpretable Affective Analysis
por: Ma, Fei, et al.
Publicado: (2026)

Enhancing Deep Learning with Optimized Gradient Descent: Bridging Numerical Methods and Neural Network Training
por: Ma, Yuhan, et al.
Publicado: (2024)

Making Batch Normalization Great in Federated Deep Learning
por: Zhong, Jike, et al.
Publicado: (2023)

Conda: Column-Normalized Adam for Training Large Language Models Faster
por: Wang, Junjie, et al.
Publicado: (2025)