Image de couverture de livre

Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Bandari, Abhinav, Yin, Lu, Hsieh, Cheng-Yu, Jaiswal, Ajay Kumar, Chen, Tianlong, Shen, Li, Krishna, Ranjay, Liu, Shiwei
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
par: Cao, Mingyu, et autres
Publié: (2024)

FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping
par: Jaiswal, Ajay, et autres
Publié: (2024)

On the Limitations of Language Targeted Pruning: Investigating the Calibration Language Impact in Multilingual LLM Pruning
par: Kurz, Simon, et autres
Publié: (2024)

IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining
par: Li, Yixiao, et autres
Publié: (2025)

GPTailor: Large Language Model Pruning Through Layer Cutting and Stitching
par: Su, Guinan, et autres
Publié: (2025)

How Many Parameters Does Your Task Really Need? Task Specific Pruning with LLM-Sieve
par: Reda, Waleed, et autres
Publié: (2025)

Junk DNA Hypothesis: Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs
par: Yin, Lu, et autres
Publié: (2023)

Dynamic Data Pruning for Automatic Speech Recognition
par: Xiao, Qiao, et autres
Publié: (2024)

On the Impact of Calibration Data in Post-training Quantization and Pruning
par: Williams, Miles, et autres
Publié: (2023)

Beware of Calibration Data for Pruning Large Language Models
par: Ji, Yixin, et autres
Publié: (2024)

Pruning Weights but Not Truth: Safeguarding Truthfulness While Pruning LLMs
par: Fu, Yao, et autres
Publié: (2025)

Think Before You Prune: Selective Self-Generated Calibration for Pruning Large Reasoning Models
par: Xiang, Yang, et autres
Publié: (2025)

Prune4Web: DOM Tree Pruning Programming for Web Agent
par: Zhang, Jiayuan, et autres
Publié: (2025)

The Hard Positive Truth about Vision-Language Compositionality
par: Kamath, Amita, et autres
Publié: (2024)

Iterative Structured Pruning for Large Language Models with Multi-Domain Calibration
par: Wu, Guangxin, et autres
Publié: (2026)

AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
par: He, Di, et autres
Publié: (2025)

Pruning as a Domain-specific LLM Extractor
par: Zhang, Nan, et autres
Publié: (2024)

Prune as You Generate: Online Rollout Pruning for Faster and Better RLVR
par: Xu, Haobo, et autres
Publié: (2026)

MINI-LLM: Memory-Efficient Structured Pruning for Large Language Models
par: Cheng, Hongrong, et autres
Publié: (2024)

Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruning
par: Kim, Minkyu, et autres
Publié: (2026)

MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
par: Qin, Jiayu, et autres
Publié: (2025)

Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
par: Yin, Lu, et autres
Publié: (2023)

LLM Pruning and Distillation in Practice: The Minitron Approach
par: Sreenivas, Sharath Turuvekere, et autres
Publié: (2024)

Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models
par: Li, Jianwei, et autres
Publié: (2023)

Prune&Comp: Free Lunch for Layer-Pruned LLMs via Iterative Pruning with Magnitude Compensation
par: Chen, Xinrui, et autres
Publié: (2025)

PruneCD: Contrasting Pruned Self Model to Improve Decoding Factuality
par: Yu, Byeongho, et autres
Publié: (2025)

Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning
par: Huang, Xijie, et autres
Publié: (2023)

OBCache: Optimal Brain KV Cache Pruning for Efficient Long-Context LLM Inference
par: Gu, Yuzhe, et autres
Publié: (2025)

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
par: Hsieh, Cheng-Yu, et autres
Publié: (2024)

O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
par: Luo, Haotian, et autres
Publié: (2025)

High-Fidelity Pruning for Large Language Models
par: Zhu, Yijun, et autres
Publié: (2026)

Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?
par: Li, Pengxiang, et autres
Publié: (2026)

DISP-LLM: Dimension-Independent Structural Pruning for Large Language Models
par: Gao, Shangqian, et autres
Publié: (2024)

Structural Pruning of Large Vision Language Models: A Comprehensive Study on Pruning Dynamics, Recovery, and Data Efficiency
par: Huang, Yiran, et autres
Publié: (2026)

IG-Pruning: Input-Guided Block Pruning for Large Language Models
par: Qiao, Kangyu, et autres
Publié: (2025)

Prune-Quantize-Distill: An Ordered Pipeline for Efficient Neural Network Compression
par: Zhou, Longsheng, et autres
Publié: (2026)

Improving Generalization in LLM Structured Pruning via Function-Aware Neuron Grouping
par: Yu, Tao, et autres
Publié: (2025)

STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning
par: Lee, Jaeseong, et autres
Publié: (2024)

Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization
par: Chrysostomou, George, et autres
Publié: (2023)

You Only Prune Once: Designing Calibration-Free Model Compression With Policy Learning
par: Sengupta, Ayan, et autres
Publié: (2025)