Copertina

Activation Sparsity Opportunities for Compressing General Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Dhar, Nobel, Deng, Bobin, Islam, Md Romyull, Nasif, Kazi Fahim Ahmad, Zhao, Liang, Suo, Kun
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

A Sparsity Predicting Approach for Large Language Models via Activation Pattern Clustering
di: Dhar, Nobel, et al.
Pubblicazione: (2025)

Characterizing and Understanding Energy Footprint and Efficiency of Small Language Model on Edges
di: Islam, Md Romyull, et al.
Pubblicazione: (2025)

SoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large Language Model Compression
di: Huang, Xinhao, et al.
Pubblicazione: (2026)

Training-Free Activation Sparsity in Large Language Models
di: Liu, James, et al.
Pubblicazione: (2024)

Large Language Model Compression with Global Rank and Sparsity Optimization
di: Zhou, Changhai, et al.
Pubblicazione: (2025)

Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity
di: Yan, Guang, et al.
Pubblicazione: (2025)

LiPCoT: Linear Predictive Coding based Tokenizer for Self-supervised Learning of Time Series Data via Language Models
di: Anjum, Md Fahim
Pubblicazione: (2024)

LLM-ProS: Analyzing Large Language Models' Performance in Competitive Problem Solving
di: Hossain, Md Sifat, et al.
Pubblicazione: (2025)

Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
di: An, Tai, et al.
Pubblicazione: (2025)

Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression
di: Zhang, Te, et al.
Pubblicazione: (2025)

Diffusion-Driven Generative Framework for Molecular Conformation Prediction
di: Yang, Bobin, et al.
Pubblicazione: (2023)

ELAS: Efficient Pre-Training of Low-Rank Large Language Models via 2:4 Activation Sparsity
di: Li, Jiaxi, et al.
Pubblicazione: (2026)

A-ProS: Towards Reliable Autonomous Programming Through Multi-Model Feedback
di: Tabassum, Anika, et al.
Pubblicazione: (2026)

Large Language Models for Education and Research: An Empirical and User Survey-based Analysis
di: Rahman, Md Mostafizer, et al.
Pubblicazione: (2025)

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
di: Song, Chenyang, et al.
Pubblicazione: (2024)

Inference Time Context Sparsity: Illusion or Opportunity?
di: Joshi, Sahil, et al.
Pubblicazione: (2026)

Large Language Models for Robotics: Opportunities, Challenges, and Perspectives
di: Wang, Jiaqi, et al.
Pubblicazione: (2024)

Small Language Models for Efficient Agentic Tool Calling: Outperforming Large Models with Targeted Fine-tuning
di: Jhandi, Polaris, et al.
Pubblicazione: (2025)

Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation
di: Yang, Hao, et al.
Pubblicazione: (2024)

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
di: Cheng, Xin, et al.
Pubblicazione: (2026)

ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching
di: Zhao, Youpeng, et al.
Pubblicazione: (2024)

Solving Generalized Grouping Problems in Cellular Manufacturing Systems Using a Network Flow Model
di: Uddin, Md. Kutub, et al.
Pubblicazione: (2024)

Development and Validation of a Large Language Model for Generating Fully-Structured Radiology Reports
di: Niu, Chuang, et al.
Pubblicazione: (2024)

GWT: Scalable Optimizer State Compression for Large Language Model Training
di: Wen, Ziqing, et al.
Pubblicazione: (2025)

Large Language Models for Financial Aid in Financial Time-series Forecasting
di: Islam, Md Khairul, et al.
Pubblicazione: (2024)

FedMTFI: Feature Importance Based Optimized Multi Teacher Knowledge Distillation in Heterogeneous Federated Learning Environment
di: Shadin, Nazmus Shakib, et al.
Pubblicazione: (2026)

Sparsity Induction for Accurate Post-Training Pruning of Large Language Models
di: Jiang, Minhao, et al.
Pubblicazione: (2026)

Generating Energy-Efficient Code via Large-Language Models -- Where are we now?
di: Apsan, Radu, et al.
Pubblicazione: (2025)

OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models
di: Deng, Yuchen, et al.
Pubblicazione: (2026)

Advancing Diffusion Models: Alias-Free Resampling and Enhanced Rotational Equivariance
di: Anjum, Md Fahim
Pubblicazione: (2024)

WhisQ: Cross-Modal Representation Learning for Text-to-Music MOS Prediction
di: Emon, Jakaria Islam, et al.
Pubblicazione: (2025)

The Collaboration Paradox: Why Generative AI Requires Both Strategic Intelligence and Operational Stability in Supply Chain Management
di: Dhar, Soumyadeep
Pubblicazione: (2025)

Exploring the Benefit of Activation Sparsity in Pre-training
di: Zhang, Zhengyan, et al.
Pubblicazione: (2024)

The Robustness of Spiking Neural Networks in Communication and its Application towards Network Efficiency in Federated Learning
di: Nguyen, Manh V., et al.
Pubblicazione: (2024)

Learn To be Efficient: Build Structured Sparsity in Large Language Models
di: Zheng, Haizhong, et al.
Pubblicazione: (2024)

DateLogicQA: Benchmarking Temporal Biases in Large Language Models
di: Bhatia, Gagan, et al.
Pubblicazione: (2024)

WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inference
di: Chen, Sihan, et al.
Pubblicazione: (2025)

One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models
di: Shao, Hang, et al.
Pubblicazione: (2023)

Embedding with Large Language Models for Classification of HIPAA Safeguard Compliance Rules
di: Rahman, Md Abdur, et al.
Pubblicazione: (2024)

Ethic-BERT: An Enhanced Deep Learning Model for Ethical and Non-Ethical Content Classification
di: Mahadi, Mahamodul Hasan, et al.
Pubblicazione: (2025)