Cover Image

Scaling Law for Quantization-Aware Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Mengzhao, Zhang, Chaoyi, Liu, Jing, Zeng, Yutao, Xue, Zeyue, Liu, Zhiheng, Li, Yunshui, Ma, Jin, Huang, Jie, Zhou, Xun, Luo, Ping
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formats
by: Chen, Mengzhao, et al.
Published: (2025)

EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
by: Chen, Mengzhao, et al.
Published: (2024)

PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization
by: Chen, Mengzhao, et al.
Published: (2024)

Scaling Laws for Floating Point Quantization Training
by: Sun, Xingwu, et al.
Published: (2025)

Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models
by: Wang, Ya, et al.
Published: (2025)

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
by: Zheng, Chen, et al.
Published: (2025)

Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models
by: Zhou, Chenxi, et al.
Published: (2025)

Scaling Laws for Post Training Quantized Large Language Models
by: Xu, Zifei, et al.
Published: (2024)

Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling
by: Huang, Hongzhi, et al.
Published: (2025)

DanceGRPO: Unleashing GRPO on Visual Generation
by: Xue, Zeyue, et al.
Published: (2025)

Efficient Pretraining Length Scaling
by: Wu, Bohong, et al.
Published: (2025)

Model Merging in Pre-training of Large Language Models
by: Li, Yunshui, et al.
Published: (2025)

Scaling Laws For Mixed Quantization
by: Cao, Zeyu, et al.
Published: (2024)

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
by: Ouyang, Xu, et al.
Published: (2024)

SASQ: Static Activation Scaling for Quantization-Aware Training in Large Language Models
by: Mao, Shizhuo, et al.
Published: (2025)

Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models
by: Chen, Longze, et al.
Published: (2024)

OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
by: Shao, Wenqi, et al.
Published: (2023)

ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization
by: Liu, Zechun, et al.
Published: (2025)

WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception
by: Liu, Zhiheng, et al.
Published: (2025)

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization
by: Zhuo, Zhijian, et al.
Published: (2025)

A Systematic Post-Train Framework for Video Generation
by: Xue, Zeyue, et al.
Published: (2026)

InfoLaw: Information Scaling Laws for Large Language Models with Quality-Weighted Mixture Data and Repetition
by: Liu, Fengze, et al.
Published: (2026)

P$^2$ Law: Scaling Law for Post-Training After Model Pruning
by: Chen, Xiaodong, et al.
Published: (2024)

D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models
by: Que, Haoran, et al.
Published: (2024)

Compression Scaling Laws:Unifying Sparsity and Quantization
by: Frantar, Elias, et al.
Published: (2025)

Parallel Loop Transformer for Efficient Test-Time Computation Scaling
by: Wu, Bohong, et al.
Published: (2025)

Marathon: A Race Through the Realm of Long Context with Large Language Models
by: Zhang, Lei, et al.
Published: (2023)

Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models
by: Zhuo, Zhijian, et al.
Published: (2024)

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
by: Tao, Chaofan, et al.
Published: (2024)

ScholarCopilot: Training Large Language Models for Academic Writing with Accurate Citations
by: Wang, Yubo, et al.
Published: (2025)

Ruler: A Model-Agnostic Method to Control Generated Length for Large Language Models
by: Li, Jiaming, et al.
Published: (2024)

Frac-Connections: Fractional Extension of Hyper-Connections
by: Zhu, Defa, et al.
Published: (2025)

STORYTELLER: An Enhanced Plot-Planning Framework for Coherent and Cohesive Story Generation
by: Li, Jiaming, et al.
Published: (2025)

FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
by: Lai, Xunhao, et al.
Published: (2025)

Densing Law of LLMs
by: Xiao, Chaojun, et al.
Published: (2024)

TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Fly
by: Koike-Akino, Toshiaki, et al.
Published: (2026)

Low-Rank Quantization-Aware Training for LLMs
by: Bondarenko, Yelysei, et al.
Published: (2024)

Scaling Laws for Code: Every Programming Language Matters
by: Yang, Jian, et al.
Published: (2025)

Parallel Scaling Law for Language Models
by: Chen, Mouxiang, et al.
Published: (2025)

Balanced Actor Initialization: Stable RLHF Training of Distillation-Based Reasoning Models
by: Zheng, Chen, et al.
Published: (2025)