Cover Image

Compression Scaling Laws:Unifying Sparsity and Quantization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Frantar, Elias, Evci, Utku, Park, Wonpyo, Houlsby, Neil, Alistarh, Dan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Extreme Compression of Large Language Models via Additive Quantization
by: Egiazarian, Vage, et al.
Published: (2024)

The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws
by: Jin, Tian, et al.
Published: (2025)

Unified Scaling Laws for Compressed Representations
by: Panferov, Andrei, et al.
Published: (2025)

SPADE: Sparsity-Guided Debugging for Deep Neural Networks
by: Moakhar, Arshia Soltani, et al.
Published: (2023)

Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization
by: Son, Seungwoo, et al.
Published: (2024)

Error Feedback Can Accurately Compress Preconditioners
by: Modoranu, Ionut-Vlad, et al.
Published: (2023)

Dynamic Sparse Training with Structured Sparsity
by: Lasby, Mike, et al.
Published: (2023)

Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimization
by: Shin, Sungbin, et al.
Published: (2024)

ECO: Quantized Training without Full-Precision Master Weights
by: Nikdan, Mahdi, et al.
Published: (2026)

Scaling Laws For Mixed Quantization
by: Cao, Zeyu, et al.
Published: (2024)

Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
by: Nrusimha, Aniruddha, et al.
Published: (2024)

Efficient Data Selection at Scale via Influence Distillation
by: Nikdan, Mahdi, et al.
Published: (2025)

Scaling Law for Quantization-Aware Training
by: Chen, Mengzhao, et al.
Published: (2025)

MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models
by: Frantar, Elias, et al.
Published: (2024)

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling
by: Dadgarnia, Alireza, et al.
Published: (2026)

Scaling Laws for Post Training Quantized Large Language Models
by: Xu, Zifei, et al.
Published: (2024)

HUKUKBERT: Domain-Specific Language Model for Turkish Law
by: Öztürk, Mehmet Utku, et al.
Published: (2026)

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
by: Ouyang, Xu, et al.
Published: (2024)

Scaling Laws for Floating Point Quantization Training
by: Sun, Xingwu, et al.
Published: (2025)

MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning
by: Modoranu, Ionut-Vlad, et al.
Published: (2026)

Task-Circuit Quantization: Leveraging Knowledge Localization and Interpretability for Compression
by: Xiao, Hanqi, et al.
Published: (2025)

DarwinLM: Evolutionary Structured Pruning of Large Language Models
by: Tang, Shengkun, et al.
Published: (2025)

Conditional Diffusion on Web-Scale Image Pairs leads to Diverse Image Variations
by: Kumar, Manoj, et al.
Published: (2024)

RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation
by: Nikdan, Mahdi, et al.
Published: (2024)

Distillation Scaling Laws
by: Busbridge, Dan, et al.
Published: (2025)

Unifying Learning Dynamics and Generalization in Transformers Scaling Law
by: Yang, Chiwun
Published: (2025)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
by: Bethune, Louis, et al.
Published: (2025)

Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on Large Language Models
by: Kurtic, Eldar, et al.
Published: (2024)

Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models
by: Zhou, Chenxi, et al.
Published: (2025)

Scaling Laws for Precision
by: Kumar, Tanishq, et al.
Published: (2024)

CRVQ: Channel-Relaxed Vector Quantization for Extreme Compression of LLMs
by: Xu, Yuzhuang, et al.
Published: (2024)

Neural Neural Scaling Laws
by: Hu, Michael Y., et al.
Published: (2026)

On the Compressibility of Quantized Large Language Models
by: Mao, Yu, et al.
Published: (2024)

Foundations of Large Language Model Compression -- Part 1: Weight Quantization
by: Young, Sean I.
Published: (2024)

Activation Quantization of Vision Encoders Needs Prefixing Registers
by: Kim, Seunghyeon, et al.
Published: (2025)

Entropy Law: The Story Behind Data Compression and LLM Performance
by: Yin, Mingjia, et al.
Published: (2024)

On the Importance of a Multi-Scale Calibration for Quantization
by: Son, Seungwoo, et al.
Published: (2026)

Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recall
by: Wang, Qianli, et al.
Published: (2025)

Towards Robust Scaling Laws for Optimizers
by: Volkova, Alexandra, et al.
Published: (2026)

Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
by: Luo, Yuqi, et al.
Published: (2024)