Imagen de Portada

GRASP: Replace Redundant Layers with Adaptive Singular Parameters for Efficient Model Compression

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Liu, Kainan, Zhang, Yong, Cheng, Ning, Li, Zhitao, Wang, Shaojun, Xiao, Jing
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models
por: Huang, Yanwen, et al.
Publicado: (2025)

Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression
por: Wang, Jingcun, et al.
Publicado: (2024)

Astra: Activation-Space Tail-Eigenvector Low-Rank Adaptation of Large Language Models
por: Liu, Kainan, et al.
Publicado: (2026)

SVFit: Parameter-Efficient Fine-Tuning of Large Pre-Trained Models Using Singular Values
por: Sun, Chengwei, et al.
Publicado: (2024)

xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction
por: Chang, Chi-Chih, et al.
Publicado: (2025)

Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry
por: Li, Zhuochun, et al.
Publicado: (2026)

CAMERA: Multi-Matrix Joint Compression for MoE Models via Micro-Expert Redundancy Analysis
por: Xu, Yuzhuang, et al.
Publicado: (2025)

Replacing Language Model for Style Transfer
por: Cheng, Pengyu, et al.
Publicado: (2022)

SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression
por: Wang, Xin, et al.
Publicado: (2024)

Iterative Layer-wise Distillation for Efficient Compression of Large Language Models
por: Kovalev, Grigory, et al.
Publicado: (2025)

ExpThink: Experience-Guided Reinforcement Learning for Adaptive Chain-of-Thought Compression
por: Bian, Tingcheng, et al.
Publicado: (2026)

SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors
por: Lingam, Vijay, et al.
Publicado: (2024)

ILRe: Intermediate Layer Retrieval for Context Compression in Causal Language Models
por: Liang, Manlai, et al.
Publicado: (2025)

DASH: Input-Aware Dynamic Layer Skipping for Efficient LLM Inference with Markov Decision Policies
por: Yang, Ning, et al.
Publicado: (2025)

SARA: Singular-Value Based Adaptive Low-Rank Adaption
por: Gu, Jihao, et al.
Publicado: (2024)

GRASP: A Rehearsal Policy for Efficient Online Continual Learning
por: Harun, Md Yousuf, et al.
Publicado: (2023)

Parameter-Efficient Fine-Tuning via Circular Convolution
por: Chen, Aochuan, et al.
Publicado: (2024)

$\textbf{AGT$^{AO}$}$: Robust and Stabilized LLM Unlearning via Adversarial Gating Training with Adaptive Orthogonality
por: Li, Pengyu, et al.
Publicado: (2026)

LongFlow: Efficient KV Cache Compression for Reasoning Models
por: Su, Yi, et al.
Publicado: (2026)

Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM
por: Solgi, Ryan, et al.
Publicado: (2025)

AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models
por: Liu, Zeyu, et al.
Publicado: (2024)

Adaptive Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization
por: Ji, Yixin, et al.
Publicado: (2024)

SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning
por: Liu, Yongkang, et al.
Publicado: (2026)

Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models
por: Chen, Yuyan, et al.
Publicado: (2024)

PEANuT: Parameter-Efficient Adaptation with Weight-aware Neural Tweakers
por: Zhong, Yibo, et al.
Publicado: (2024)

Better Prompt Compression Without Multi-Layer Perceptrons
por: Honig, Edouardo, et al.
Publicado: (2025)

Is Parameter Collision Hindering Continual Learning in LLMs?
por: Yang, Shuo, et al.
Publicado: (2024)

Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning
por: Xu, Jing, et al.
Publicado: (2024)

QET: Enhancing Quantized LLM Parameters and KV cache Compression through Element Substitution and Residual Clustering
por: Wang, Yanshu, et al.
Publicado: (2024)

Streamlining Redundant Layers to Compress Large Language Models
por: Chen, Xiaodong, et al.
Publicado: (2024)

RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
por: Tang, Hanlin, et al.
Publicado: (2024)

Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models
por: Yao, Kai, et al.
Publicado: (2024)

LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models
por: Li, Guangyan, et al.
Publicado: (2024)

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
por: Lu, Xudong, et al.
Publicado: (2024)

Parameter-Efficient Fine-Tuning for Foundation Models
por: Zhang, Dan, et al.
Publicado: (2025)

Hidden Heroes and Gradient Bloats: Layer-Wise Redundancy Inverts Attribution in Transformers
por: Ye, Donald
Publicado: (2026)

Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compression
por: Zhang, Yong, et al.
Publicado: (2025)

Adaptive Layer Selection for Layer-Wise Token Pruning in LLM Inference
por: Taniguchi, Rei, et al.
Publicado: (2026)

BIPEFT: Budget-Guided Iterative Search for Parameter Efficient Fine-Tuning of Large Pretrained Language Models
por: Chang, Aofei, et al.
Publicado: (2024)

ByteFlow: Language Modeling through Adaptive Byte Compression without a Tokenizer
por: Deng, Chunyuan, et al.
Publicado: (2026)