Buchumschlag

GWT: Scalable Optimizer State Compression for Large Language Model Training

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Wen, Ziqing, Luo, Ping, Wang, Jiahuan, Yuan, Kun, Li, Dongsheng, Sun, Tao
Format:	Preprint
Veröffentlicht:	2025
Schlagworte:	Machine Learning Artificial Intelligence
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Ähnliche Einträge

FOAM: Blocked State Folding for Memory-Efficient LLM Training
von: Wen, Ziqing, et al.
Veröffentlicht: (2025)

Revealing Modular Gradient Noise Imbalance in LLMs: Calibrating Adam via Signal-to-Noise Ratio
von: Wen, Ziqing, et al.
Veröffentlicht: (2026)

Unveiling High-Probability Generalization in Decentralized SGD
von: Wang, Jiahuan, et al.
Veröffentlicht: (2026)

Stability and Generalization for Decentralized Markov SGD
von: Wang, Jiahuan, et al.
Veröffentlicht: (2026)

Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity
von: Luo, Ping, et al.
Veröffentlicht: (2026)

Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities
von: Hao, Zhiwei, et al.
Veröffentlicht: (2025)

BackSlash: Rate Constrained Optimized Training of Large Language Models
von: Wu, Jun, et al.
Veröffentlicht: (2025)

Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving
von: Gao, Wei, et al.
Veröffentlicht: (2025)

Large Language Model Compression with Global Rank and Sparsity Optimization
von: Zhou, Changhai, et al.
Veröffentlicht: (2025)

Activation Sparsity Opportunities for Compressing General Large Language Models
von: Dhar, Nobel, et al.
Veröffentlicht: (2024)

Scalable Hyperparameter-Divergent Ensemble Training with Automatic Learning Rate Exploration for Large Models
von: Cheng, Hailing, et al.
Veröffentlicht: (2026)

Score-based Generative Models with Adaptive Momentum
von: Wen, Ziqing, et al.
Veröffentlicht: (2024)

InverseScope: Scalable Activation Inversion for Interpreting Large Language Models
von: Luo, Yifan, et al.
Veröffentlicht: (2025)

OptProver: Bridging Olympiad and Optimization through Continual Training in Formal Theorem Proving
von: Li, Chenyi, et al.
Veröffentlicht: (2026)

Generalized Fisher-Weighted SVD: Scalable Kronecker-Factored Fisher Approximation for Compressing Large Language Models
von: Chekalina, Viktoriia, et al.
Veröffentlicht: (2025)

EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
von: Chen, Mengzhao, et al.
Veröffentlicht: (2024)

MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks
von: Zhang, Lei, et al.
Veröffentlicht: (2023)

In-context Autoencoder for Context Compression in a Large Language Model
von: Ge, Tao, et al.
Veröffentlicht: (2023)

D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
von: Li, Junlin, et al.
Veröffentlicht: (2026)

Self-Improved Learning for Scalable Neural Combinatorial Optimization
von: Luo, Fu, et al.
Veröffentlicht: (2024)

CrossQuant: A Post-Training Quantization Method with Smaller Quantization Kernel for Precise Large Language Model Compression
von: Liu, Wenyuan, et al.
Veröffentlicht: (2024)

SVTime: Small Time Series Forecasting Models Informed by "Physics" of Large Vision Model Forecasters
von: Shen, ChengAo, et al.
Veröffentlicht: (2025)

HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference
von: Gong, Ping, et al.
Veröffentlicht: (2025)

LLMExplainer: Large Language Model based Bayesian Inference for Graph Explanation Generation
von: Zhang, Jiaxing, et al.
Veröffentlicht: (2024)

CoSA: Compressed Sensing-Based Adaptation of Large Language Models
von: Wei, Songtao, et al.
Veröffentlicht: (2026)

Enhancing Stability for Large Language Models Training in Constrained Bandwidth Networks
von: Dai, Yun, et al.
Veröffentlicht: (2024)

Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
von: Lu, Haoran, et al.
Veröffentlicht: (2025)

Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints
von: Yang, Jing, et al.
Veröffentlicht: (2025)

GVPO: Group Variance Policy Optimization for Large Language Model Post-Training
von: Zhang, Kaichen, et al.
Veröffentlicht: (2025)

How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehension
von: Dai, Xinnan, et al.
Veröffentlicht: (2024)

Federated Prediction-Powered Inference from Decentralized Data
von: Luo, Ping, et al.
Veröffentlicht: (2024)

Scaling and Transferability of Annealing Strategies in Large Language Model Training
von: Wang, Siqi, et al.
Veröffentlicht: (2025)

Train at Moving Edge: Online-Verified Prompt Selection for Efficient RL Training of Large Reasoning Model
von: Wu, Jiahao, et al.
Veröffentlicht: (2026)

On the Compressibility of Quantized Large Language Models
von: Mao, Yu, et al.
Veröffentlicht: (2024)

AtmosSci-Bench: Evaluating the Recent Advance of Large Language Model for Atmospheric Science
von: Li, Chenyue, et al.
Veröffentlicht: (2025)

Scalable Efficient Training of Large Language Models with Low-dimensional Projected Attention
von: Lv, Xingtai, et al.
Veröffentlicht: (2024)

An Interpretable and Scalable Framework for Evaluating Large Language Models
von: Qu, Xinhao, et al.
Veröffentlicht: (2026)

NoWag: A Unified Framework for Shape Preserving Compression of Large Language Models
von: Liu, Lawrence, et al.
Veröffentlicht: (2025)

PT$^2$-LLM: Post-Training Ternarization for Large Language Models
von: Yan, Xianglong, et al.
Veröffentlicht: (2025)

Exploratory Memory-Augmented LLM Agent via Hybrid On- and Off-Policy Optimization
von: Liu, Zeyuan, et al.
Veröffentlicht: (2026)