Cover Image

SoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large Language Model Compression

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Huang, Xinhao, Huang, You-Liang, Wen, Zeyi
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DeInfer: Efficient Parallel Inferencing for Decomposed Large Language Models
by: Huang, You-Liang, et al.
Published: (2026)

Training-Free Activation Sparsity in Large Language Models
by: Liu, James, et al.
Published: (2024)

SkipCat: Rank-Maximized Low-Rank Compression of Large Language Models via Shared Projection and Block Skipping
by: Lu, Yu-Chen, et al.
Published: (2025)

Leveraging Large Language Models for Entity Matching
by: Huang, Qianyu, et al.
Published: (2024)

TLoRA: Task-aware Low Rank Adaptation of Large Language Models
by: Lin, Weicheng, et al.
Published: (2026)

Leveraging Large Language Models for Scalable Vector Graphics-Driven Image Understanding
by: Cai, Mu, et al.
Published: (2023)

JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking
by: Niu, Tong, et al.
Published: (2024)

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
by: Cheng, Xin, et al.
Published: (2026)

Flexora: Flexible Low Rank Adaptation for Large Language Models
by: Wei, Chenxing, et al.
Published: (2024)

OlympicArena Medal Ranks: Who Is the Most Intelligent AI So Far?
by: Huang, Zhen, et al.
Published: (2024)

LoRTA: Low Rank Tensor Adaptation of Large Language Models
by: Hounie, Ignacio, et al.
Published: (2024)

Transitivity Meets Cyclicity: Explicit Preference Decomposition for Dynamic Large Language Model Alignment
by: Huang, Yucong, et al.
Published: (2026)

FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Model
by: Hu, Jinwei, et al.
Published: (2025)

Skewed Memorization in Large Language Models: Quantification and Decomposition
by: Li, Hao, et al.
Published: (2025)

DenseLoRA: Dense Low-Rank Adaptation of Large Language Models
by: Mu, Lin, et al.
Published: (2025)

Sparsity Induction for Accurate Post-Training Pruning of Large Language Models
by: Jiang, Minhao, et al.
Published: (2026)

LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Merging
by: Liu, Zehua, et al.
Published: (2025)

Controlled Low-Rank Adaptation with Subspace Regularization for Continued Training on Large Language Models
by: Lu, Yuheng, et al.
Published: (2024)

Exploring the Benefit of Activation Sparsity in Pre-training
by: Zhang, Zhengyan, et al.
Published: (2024)

Rank-Then-Score: Enhancing Large Language Models for Automated Essay Scoring
by: Cai, Yida, et al.
Published: (2025)

Penrose Tiled Low-Rank Compression and Section-Wise Q&A Fine-Tuning: A General Framework for Domain-Specific Large Language Model Adaptation
by: Kuo, Chuan-Wei, et al.
Published: (2025)

Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performance
by: Xu, Borui, et al.
Published: (2025)

One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models
by: Shao, Hang, et al.
Published: (2023)

LoLCATs: On Low-Rank Linearizing of Large Language Models
by: Zhang, Michael, et al.
Published: (2024)

ERC-SVD: Error-Controlled SVD for Large Language Model Compression
by: Bai, Haolei, et al.
Published: (2025)

Pruning Large Language Models to Intra-module Low-rank Architecture with Transitional Activations
by: Shen, Bowen, et al.
Published: (2024)

A Sparsity Predicting Approach for Large Language Models via Activation Pattern Clustering
by: Dhar, Nobel, et al.
Published: (2025)

Prompt Compression in Diffusion Large Language Models: Evaluating LLMLingua-2 on LLaDA
by: Huang, Sterling, et al.
Published: (2026)

When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models
by: Wang, Weilan, et al.
Published: (2025)

Training-Free Bayesianization for Low-Rank Adapters of Large Language Models
by: Shi, Haizhou, et al.
Published: (2024)

Achieving Sparse Activation in Small Language Models
by: Song, Jifeng, et al.
Published: (2024)

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
by: Song, Chenyang, et al.
Published: (2024)

Batched Low-Rank Adaptation of Foundation Models
by: Wen, Yeming, et al.
Published: (2023)

Learn To be Efficient: Build Structured Sparsity in Large Language Models
by: Zheng, Haizhong, et al.
Published: (2024)

DaMoC: Efficiently Selecting the Optimal Large Language Model for Fine-tuning Domain Tasks Based on Data and Model Compression
by: Huang, Wei, et al.
Published: (2025)

A Survey on Model Compression for Large Language Models
by: Zhu, Xunyu, et al.
Published: (2023)

R-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning
by: Shan, Hongyu, et al.
Published: (2025)

Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models
by: Ren, Qingyu, et al.
Published: (2025)

Atomic-SNLI: Fine-Grained Natural Language Inference through Atomic Fact Decomposition
by: Huang, Minghui
Published: (2026)

Activation-Informed Merging of Large Language Models
by: Nobari, Amin Heyrani, et al.
Published: (2025)