Cover Image

SparseLLM: Towards Global Pruning for Pre-trained Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bai, Guangji, Li, Yijiang, Ling, Chen, Kim, Kibaek, Zhao, Liang
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FedSpaLLM: Federated Pruning of Large Language Models
by: Bai, Guangji, et al.
Published: (2024)

Enhancing One-shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse Mechanism
by: Li, Guanchen, et al.
Published: (2024)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
by: Song, Weixi, et al.
Published: (2023)

NutePrune: Efficient Progressive Pruning with Numerous Teachers for Large Language Models
by: Li, Shengrui, et al.
Published: (2024)

PrunePath: Towards Highly Structured Sparse Language Models
by: Gu, Zhexuan, et al.
Published: (2026)

PAT: Pruning-Aware Tuning for Large Language Models
by: Liu, Yijiang, et al.
Published: (2024)

Structural Pruning of Pre-trained Language Models via Neural Architecture Search
by: Klein, Aaron, et al.
Published: (2024)

Enhancing Vision-Language Model Pre-training with Image-text Pair Pruning Based on Word Frequency
by: Liang, Mingliang, et al.
Published: (2024)

Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
by: Xia, Mengzhou, et al.
Published: (2023)

Can Pre-trained Language Models Understand Chinese Humor?
by: Chen, Yuyan, et al.
Published: (2024)

MediSwift: Efficient Sparse Pre-trained Biomedical Language Models
by: Thangarasa, Vithursan, et al.
Published: (2024)

SEAP: Training-free Sparse Expert Activation Pruning Unlock the Brainpower of Large Language Models
by: Liang, Xun, et al.
Published: (2025)

Model Merging in Pre-training of Large Language Models
by: Li, Yunshui, et al.
Published: (2025)

StructPrune: Structured Global Pruning asymptotics with $\mathcal{O}(\sqrt{N})$ GPU Memory
by: Song, Xinyuan, et al.
Published: (2025)

Examining Forgetting in Continual Pre-training of Aligned Large Language Models
by: Li, Chen-An, et al.
Published: (2024)

Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization
by: Zhang, Haode, et al.
Published: (2022)

Uncertainty Quantification for In-Context Learning of Large Language Models
by: Ling, Chen, et al.
Published: (2024)

Spoken Language Identification with Pre-trained Models and Margin Loss
by: Fang, Zhihua, et al.
Published: (2026)

RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuning
by: Wang, Haoyu, et al.
Published: (2024)

TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models
by: Yan, Junbing, et al.
Published: (2024)

Towards Effective and Efficient Continual Pre-training of Large Language Models
by: Chen, Jie, et al.
Published: (2024)

Exploiting Vocabulary Frequency Imbalance in Language Model Pre-training
by: Chung, Woojin, et al.
Published: (2025)

Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models
by: He, Yu, et al.
Published: (2025)

Metadata Conditioning Accelerates Language Model Pre-training
by: Gao, Tianyu, et al.
Published: (2025)

Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-training
by: Wu, Linjuan, et al.
Published: (2025)

Pruning Large Language Models with Semi-Structural Adaptive Sparse Training
by: Huang, Weiyu, et al.
Published: (2024)

Pre-training Distillation for Large Language Models: A Design Space Exploration
by: Peng, Hao, et al.
Published: (2024)

On the Limitations of Language Targeted Pruning: Investigating the Calibration Language Impact in Multilingual LLM Pruning
by: Kurz, Simon, et al.
Published: (2024)

Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models
by: Li, Jianwei, et al.
Published: (2023)

Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
by: Qian, Chen, et al.
Published: (2024)

Alignment at Pre-training! Towards Native Alignment for Arabic LLMs
by: Liang, Juhao, et al.
Published: (2024)

OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration
by: Wang, Shaobo, et al.
Published: (2026)

UOR: Universal Backdoor Attacks on Pre-trained Language Models
by: Du, Wei, et al.
Published: (2023)

KoCo: Conditioning Language Model Pre-training on Knowledge Coordinates
by: Li, Yudong, et al.
Published: (2026)

Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
by: Tu, Zhijun, et al.
Published: (2025)

Development of Cognitive Intelligence in Pre-trained Language Models
by: Shah, Raj Sanjay, et al.
Published: (2024)

E$^3$-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models
by: Yuan, Tao, et al.
Published: (2025)

Evaluating Discourse Cohesion in Pre-trained Language Models
by: He, Jie, et al.
Published: (2025)

Probing Language Models for Pre-training Data Detection
by: Liu, Zhenhua, et al.
Published: (2024)

AfriqueLLM: How Data Mixing and Model Architecture Impact Continued Pre-training for African Languages
by: Yu, Hao, et al.
Published: (2026)