Cover Image

AVSS: Layer Importance Evaluation in Large Language Models via Activation Variance-Sparsity Analysis

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Song, Zichen, Wu, Yuxin, Huang, Sitan, Kang, Zhongfeng
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Layer Importance and Hallucination Analysis in Large Language Models via Enhanced Activation Variance-Sparsity
by: Song, Zichen, et al.
Published: (2024)

EM-MIAs: Enhancing Membership Inference Attacks in Large Language Models through Ensemble Modeling
by: Song, Zichen, et al.
Published: (2024)

ActTail: Global Activation Sparsity in Large Language Models
by: Hou, Wenwen, et al.
Published: (2026)

Training-Free Activation Sparsity in Large Language Models
by: Liu, James, et al.
Published: (2024)

Investigating Layer Importance in Large Language Models
by: Zhang, Yang, et al.
Published: (2024)

SoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large Language Model Compression
by: Huang, Xinhao, et al.
Published: (2026)

Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
by: Luo, Yuqi, et al.
Published: (2024)

Beyond Spurious Signals: Debiasing Multimodal Large Language Models via Counterfactual Inference and Adaptive Expert Routing
by: Wu, Zichen, et al.
Published: (2025)

Likelihood Variance as Text Importance for Resampling Texts to Map Language Models
by: Oyama, Momose, et al.
Published: (2025)

A Sparsity Predicting Approach for Large Language Models via Activation Pattern Clustering
by: Dhar, Nobel, et al.
Published: (2025)

Block Circulant Adapter for Large Language Models
by: Ding, Xinyu, et al.
Published: (2025)

Sparsity-Accelerated Training for Large Language Models
by: Ma, Da, et al.
Published: (2024)

SASQ: Static Activation Scaling for Quantization-Aware Training in Large Language Models
by: Mao, Shizhuo, et al.
Published: (2025)

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
by: Song, Chenyang, et al.
Published: (2024)

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
by: Qiu, Zihan, et al.
Published: (2025)

SURGE: On the Potential of Large Language Models as General-Purpose Surrogate Code Executors
by: Lyu, Bohan, et al.
Published: (2025)

Activation-Guided Consensus Merging for Large Language Models
by: Yao, Yuxuan, et al.
Published: (2025)

Embedding Domain Knowledge for Large Language Models via Reinforcement Learning from Augmented Generation
by: Nie, Chaojun, et al.
Published: (2025)

Towards Efficient and Effective Alignment of Large Language Models
by: Jiang, Yuxin
Published: (2025)

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
by: Cheng, Xin, et al.
Published: (2026)

CAST: Continuous and Differentiable Semi-Structured Sparsity-Aware Training for Large Language Models
by: Huang, Weiyu, et al.
Published: (2025)

ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
by: Yuan, Zhihang, et al.
Published: (2023)

A Single Layer to Explain Them All:Understanding Massive Activations in Large Language Models
by: Shi, Zeru, et al.
Published: (2026)

CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models
by: Lee, Donghyun, et al.
Published: (2024)

Attention Heads of Large Language Models: A Survey
by: Zheng, Zifan, et al.
Published: (2024)

Large Language Model Evaluation via Matrix Nuclear-Norm
by: Li, Yahan, et al.
Published: (2024)

On Importance of Layer Pruning for Smaller BERT Models and Low Resource Languages
by: Shirke, Mayur, et al.
Published: (2025)

Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activations
by: Ma, Xiaoxu, et al.
Published: (2026)

Sparsity Induction for Accurate Post-Training Pruning of Large Language Models
by: Jiang, Minhao, et al.
Published: (2026)

Layer-Condensed KV Cache for Efficient Inference of Large Language Models
by: Wu, Haoyi, et al.
Published: (2024)

Toward the Evaluation of Large Language Models Considering Score Variance across Instruction Templates
by: Sakai, Yusuke, et al.
Published: (2024)

LLM-BIP: Structured Pruning for Large Language Models with Block-Wise Forward Importance Propagation
by: Wu, Haihang
Published: (2024)

SafetyBench: Evaluating the Safety of Large Language Models
by: Zhang, Zhexin, et al.
Published: (2023)

Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models
by: Yao, Kai, et al.
Published: (2024)

GRASS: Gradient-based Adaptive Layer-wise Importance Sampling for Memory-efficient Large Language Model Fine-tuning
by: Tian, Kaiyuan, et al.
Published: (2026)

ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coverage
by: Lee, Taewhoo, et al.
Published: (2024)

A Comprehensive Evaluation of Large Language Models on Aspect-Based Sentiment Analysis
by: Zhou, Changzhi, et al.
Published: (2024)

E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsity
by: Li, Yun, et al.
Published: (2023)

Exploring the Benefit of Activation Sparsity in Pre-training
by: Zhang, Zhengyan, et al.
Published: (2024)

Learn To be Efficient: Build Structured Sparsity in Large Language Models
by: Zheng, Haizhong, et al.
Published: (2024)