Image de couverture de livre

Prune-Quantize-Distill: An Ordered Pipeline for Efficient Neural Network Compression

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhou, Longsheng, Shen, Yu
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

An Efficient Sparse Fine-Tuning with Low Quantization Error via Neural Network Pruning
par: Li, Cen-Jhih, et autres
Publié: (2025)

SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models
par: Zhu, Hourun, et autres
Publié: (2025)

Efficient Mathematical Reasoning Models via Dynamic Pruning and Knowledge Distillation
par: Yu, Fengming, et autres
Publié: (2025)

LLM Pruning and Distillation in Practice: The Minitron Approach
par: Sreenivas, Sharath Turuvekere, et autres
Publié: (2024)

IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining
par: Li, Yixiao, et autres
Publié: (2025)

On the Compressibility of Quantized Large Language Models
par: Mao, Yu, et autres
Publié: (2024)

PPC-GPT: Federated Task-Specific Compression of Large Language Models via Pruning and Chain-of-Thought Distillation
par: Fan, Tao, et autres
Publié: (2025)

Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compression
par: Qu, Xiaoyi, et autres
Publié: (2025)

ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization
par: Yadav, Prateek, et autres
Publié: (2023)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
par: Kim, Junhyuck, et autres
Publié: (2026)

Compact Language Models via Pruning and Knowledge Distillation
par: Muralidharan, Saurav, et autres
Publié: (2024)

SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
par: Tang, Shengkun, et autres
Publié: (2026)

Structural Rationale Distillation via Reasoning Space Compression
par: Yang, Jialin, et autres
Publié: (2026)

KVSculpt: KV Cache Compression as Distillation
par: Jiang, Bo, et autres
Publié: (2026)

Efficient Post-training Quantization with FP8 Formats
par: Shen, Haihao, et autres
Publié: (2023)

Quantization Dominates Rank Reduction for KV-Cache Compression
par: Salfati, Samuel
Publié: (2026)

Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting
par: Liu, Chi, et autres
Publié: (2026)

Task-Circuit Quantization: Leveraging Knowledge Localization and Interpretability for Compression
par: Xiao, Hanqi, et autres
Publié: (2025)

Pruning Foundation Models for High Accuracy without Retraining
par: Zhao, Pu, et autres
Publié: (2024)

Earley-Driven Dynamic Pruning for Efficient Structured Decoding
par: Sun, Xintong, et autres
Publié: (2025)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
par: Chen, Yingfa, et autres
Publié: (2026)

BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation
par: Xu, Peng, et autres
Publié: (2024)

I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
par: Hu, Xing, et autres
Publié: (2024)

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit
par: Gong, Ruihao, et autres
Publié: (2024)

Enhancing Delta Compression in LLMs via SVD-based Quantization Error Minimization
par: Xiong, Boya, et autres
Publié: (2025)

Projected Compression: Trainable Projection for Efficient Transformer Compression
par: Stefaniak, Maciej, et autres
Publié: (2025)

Sink-Aware Pruning for Diffusion Language Models
par: Myrzakhan, Aidar, et autres
Publié: (2026)

Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars
par: Wu, Zhaoxuan, et autres
Publié: (2024)

Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generation
par: Dong, Harry, et autres
Publié: (2024)

Text Quality-Based Pruning for Efficient Training of Language Models
par: Sharma, Vasu, et autres
Publié: (2024)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
par: Lu, Xudong, et autres
Publié: (2024)

TabDistill: Distilling Transformers into Neural Nets for Few-Shot Tabular Classification
par: Dissanayake, Pasan, et autres
Publié: (2025)

Efficiently Distilling LLMs for Edge Applications
par: Kundu, Achintya, et autres
Publié: (2024)

EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
par: Chen, Mengzhao, et autres
Publié: (2024)

Recovered in Translation: Efficient Pipeline for Automated Translation of Benchmarks and Datasets
par: Yukhymenko, Hanna, et autres
Publié: (2026)

LeanK: Learnable K Cache Channel Pruning for Efficient Decoding
par: Zhang, Yike, et autres
Publié: (2025)

Pruning Large Language Models by Identifying and Preserving Functional Networks
par: Liu, Yiheng, et autres
Publié: (2025)

EPSD: Early Pruning with Self-Distillation for Efficient Model Compression
par: Chen, Dong, et autres
Publié: (2024)

LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
par: Fu, Qichen, et autres
Publié: (2024)

Adapt-Pruner: Adaptive Structural Pruning for Efficient Small Language Model Training
par: Pan, Rui, et autres
Publié: (2025)