Cover Image

Learn To be Efficient: Build Structured Sparsity in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zheng, Haizhong, Bai, Xiaoyan, Liu, Xueshen, Mao, Z. Morley, Chen, Beidi, Lai, Fan, Prakash, Atul
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Plato: Plan to Efficiently Decode for Large Language Model Inference
by: Jin, Shuowei, et al.
Published: (2024)

AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs
by: Zheng, Haizhong, et al.
Published: (2026)

RLBoost: Harvesting Preemptible Resources for Cost-Efficient Reinforcement Learning on LLMs
by: Wu, Yongji, et al.
Published: (2025)

A Two-Stage Proactive Dialogue Generator for Efficient Clinical Information Collection Using Large Language Model
by: Li, Xueshen, et al.
Published: (2024)

Class-Proportional Coreset Selection for Difficulty-Separable Data
by: Tsai, Elisa, et al.
Published: (2025)

Efficient Streaming Language Models with Attention Sinks
by: Xiao, Guangxuan, et al.
Published: (2023)

Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?
by: Zheng, Haizhong, et al.
Published: (2025)

Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generation
by: Dong, Harry, et al.
Published: (2024)

Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
by: Zheng, Haizhong, et al.
Published: (2025)

XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models
by: Dong, Yixin, et al.
Published: (2024)

Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training
by: Hu, Pingbang, et al.
Published: (2026)

Jackpot: Optimal Budgeted Rejection Sampling for Extreme Actor-Policy Mismatch Reinforcement Learning
by: Chen, Zhuoming, et al.
Published: (2026)

Training-Free Activation Sparsity in Large Language Models
by: Liu, James, et al.
Published: (2024)

Sparsity Induction for Accurate Post-Training Pruning of Large Language Models
by: Jiang, Minhao, et al.
Published: (2026)

Sirius: Contextual Sparsity with Correction for Efficient LLMs
by: Zhou, Yang, et al.
Published: (2024)

BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation
by: Xu, Peng, et al.
Published: (2024)

Assessing Large Language Models for Online Extremism Research: Identification, Explanation, and New Knowledge
by: Dong, Beidi, et al.
Published: (2024)

MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
by: Fang, Gongfan, et al.
Published: (2024)

Dependency-Aware Semi-Structured Sparsity of GLU Variants in Large Language Models
by: Guo, Zhiyu, et al.
Published: (2024)

One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models
by: Shao, Hang, et al.
Published: (2023)

Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimation
by: Liu, Jingyu, et al.
Published: (2025)

Fine-grainedly Synthesize Streaming Data Based On Large Language Models With Graph Structure Understanding For Data Sparsity
by: Zhang, Xin, et al.
Published: (2024)

GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models
by: Li, Shilong, et al.
Published: (2024)

Long Exposure: Accelerating Parameter-Efficient Fine-Tuning for LLMs under Shadowy Sparsity
by: Wang, Tuowei, et al.
Published: (2025)

Enhancing Large Language Models through Structured Reasoning
by: Dong, Yubo, et al.
Published: (2025)

Building Guardrails for Large Language Models
by: Dong, Yi, et al.
Published: (2024)

E^2-LLM: Efficient and Extreme Length Extension of Large Language Models
by: Liu, Jiaheng, et al.
Published: (2024)

What Kind of Reasoning (if any) is an LLM actually doing? On the Stochastic Nature and Abductive Appearance of Large Language Models
by: Floridi, Luciano, et al.
Published: (2025)

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
by: Cheng, Xin, et al.
Published: (2026)

Sensitivity Meets Sparsity: The Impact of Extremely Sparse Parameter Patterns on Theory-of-Mind of Large Language Models
by: Wu, Yuheng, et al.
Published: (2025)

LLM-Barber: Block-Aware Rebuilder for Sparsity Mask in One-Shot for Large Language Models
by: Su, Yupeng, et al.
Published: (2024)

ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models
by: Akhauri, Yash, et al.
Published: (2024)

Kinetics: Rethinking Test-Time Scaling Laws
by: Sadhukhan, Ranajoy, et al.
Published: (2025)

Efficient Large Language Models: A Survey
by: Wan, Zhongwei, et al.
Published: (2023)

Uncovering Implicit Bias in Large Language Models with Concept Learning Dataset
by: Wang, Leroy Z.
Published: (2025)

Dynamic Strategy Planning for Efficient Question Answering with Large Language Models
by: Parekh, Tanmay, et al.
Published: (2024)

ARS: Adaptive Reasoning Suppression for Efficient Large Reasoning Language Models
by: Zheng, Dongqi
Published: (2025)

Medical Reasoning with Large Language Models: A Survey and MR-Bench
by: Ren, Xiaohan, et al.
Published: (2026)

Efficient Post-Training Refinement of Latent Reasoning in Large Language Models
by: Wang, Xinyuan, et al.
Published: (2025)

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
by: Song, Chenyang, et al.
Published: (2024)