Cover Image

Activation Sparsity Opportunities for Compressing General Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dhar, Nobel, Deng, Bobin, Islam, Md Romyull, Nasif, Kazi Fahim Ahmad, Zhao, Liang, Suo, Kun
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Sparsity Predicting Approach for Large Language Models via Activation Pattern Clustering
by: Dhar, Nobel, et al.
Published: (2025)

Characterizing and Understanding Energy Footprint and Efficiency of Small Language Model on Edges
by: Islam, Md Romyull, et al.
Published: (2025)

SoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large Language Model Compression
by: Huang, Xinhao, et al.
Published: (2026)

Training-Free Activation Sparsity in Large Language Models
by: Liu, James, et al.
Published: (2024)

Large Language Model Compression with Global Rank and Sparsity Optimization
by: Zhou, Changhai, et al.
Published: (2025)

Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity
by: Yan, Guang, et al.
Published: (2025)

LiPCoT: Linear Predictive Coding based Tokenizer for Self-supervised Learning of Time Series Data via Language Models
by: Anjum, Md Fahim
Published: (2024)

LLM-ProS: Analyzing Large Language Models' Performance in Competitive Problem Solving
by: Hossain, Md Sifat, et al.
Published: (2025)

Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
by: An, Tai, et al.
Published: (2025)

Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression
by: Zhang, Te, et al.
Published: (2025)

Diffusion-Driven Generative Framework for Molecular Conformation Prediction
by: Yang, Bobin, et al.
Published: (2023)

ELAS: Efficient Pre-Training of Low-Rank Large Language Models via 2:4 Activation Sparsity
by: Li, Jiaxi, et al.
Published: (2026)

A-ProS: Towards Reliable Autonomous Programming Through Multi-Model Feedback
by: Tabassum, Anika, et al.
Published: (2026)

Large Language Models for Education and Research: An Empirical and User Survey-based Analysis
by: Rahman, Md Mostafizer, et al.
Published: (2025)

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
by: Song, Chenyang, et al.
Published: (2024)

Inference Time Context Sparsity: Illusion or Opportunity?
by: Joshi, Sahil, et al.
Published: (2026)

Large Language Models for Robotics: Opportunities, Challenges, and Perspectives
by: Wang, Jiaqi, et al.
Published: (2024)

Small Language Models for Efficient Agentic Tool Calling: Outperforming Large Models with Targeted Fine-tuning
by: Jhandi, Polaris, et al.
Published: (2025)

Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation
by: Yang, Hao, et al.
Published: (2024)

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
by: Cheng, Xin, et al.
Published: (2026)

ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching
by: Zhao, Youpeng, et al.
Published: (2024)

Solving Generalized Grouping Problems in Cellular Manufacturing Systems Using a Network Flow Model
by: Uddin, Md. Kutub, et al.
Published: (2024)

Development and Validation of a Large Language Model for Generating Fully-Structured Radiology Reports
by: Niu, Chuang, et al.
Published: (2024)

GWT: Scalable Optimizer State Compression for Large Language Model Training
by: Wen, Ziqing, et al.
Published: (2025)

Large Language Models for Financial Aid in Financial Time-series Forecasting
by: Islam, Md Khairul, et al.
Published: (2024)

FedMTFI: Feature Importance Based Optimized Multi Teacher Knowledge Distillation in Heterogeneous Federated Learning Environment
by: Shadin, Nazmus Shakib, et al.
Published: (2026)

Sparsity Induction for Accurate Post-Training Pruning of Large Language Models
by: Jiang, Minhao, et al.
Published: (2026)

Generating Energy-Efficient Code via Large-Language Models -- Where are we now?
by: Apsan, Radu, et al.
Published: (2025)

OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models
by: Deng, Yuchen, et al.
Published: (2026)

Advancing Diffusion Models: Alias-Free Resampling and Enhanced Rotational Equivariance
by: Anjum, Md Fahim
Published: (2024)

WhisQ: Cross-Modal Representation Learning for Text-to-Music MOS Prediction
by: Emon, Jakaria Islam, et al.
Published: (2025)

The Collaboration Paradox: Why Generative AI Requires Both Strategic Intelligence and Operational Stability in Supply Chain Management
by: Dhar, Soumyadeep
Published: (2025)

Exploring the Benefit of Activation Sparsity in Pre-training
by: Zhang, Zhengyan, et al.
Published: (2024)

The Robustness of Spiking Neural Networks in Communication and its Application towards Network Efficiency in Federated Learning
by: Nguyen, Manh V., et al.
Published: (2024)

Learn To be Efficient: Build Structured Sparsity in Large Language Models
by: Zheng, Haizhong, et al.
Published: (2024)

DateLogicQA: Benchmarking Temporal Biases in Large Language Models
by: Bhatia, Gagan, et al.
Published: (2024)

WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inference
by: Chen, Sihan, et al.
Published: (2025)

One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models
by: Shao, Hang, et al.
Published: (2023)

Embedding with Large Language Models for Classification of HIPAA Safeguard Compliance Rules
by: Rahman, Md Abdur, et al.
Published: (2024)

Ethic-BERT: An Enhanced Deep Learning Model for Ethical and Non-Ethical Content Classification
by: Mahadi, Mahamodul Hasan, et al.
Published: (2025)