Cover Image

Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Jingcun, Chen, Yu-Guang, Lin, Ing-Chao, Li, Bing, Zhang, Grace Li
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA
by: Bae, Sangmin, et al.
Published: (2024)

GRASP: Replace Redundant Layers with Adaptive Singular Parameters for Efficient Model Compression
by: Liu, Kainan, et al.
Published: (2024)

Early-Exit with Class Exclusion for Efficient Inference of Neural Networks
by: Wang, Jingcun, et al.
Published: (2023)

FlexiGPT: Pruning and Extending Large Language Models with Low-Rank Weight Sharing
by: Smith, James Seale, et al.
Published: (2025)

FoldGPT: Simple and Effective Large Language Model Compression Scheme
by: Liu, Songwei, et al.
Published: (2024)

LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models
by: Li, Guangyan, et al.
Published: (2024)

Iterative Layer-wise Distillation for Efficient Compression of Large Language Models
by: Kovalev, Grigory, et al.
Published: (2025)

Shared Global and Local Geometry of Language Model Embeddings
by: Lee, Andrew, et al.
Published: (2025)

ILRe: Intermediate Layer Retrieval for Context Compression in Causal Language Models
by: Liang, Manlai, et al.
Published: (2025)

Quantization of Large Language Models with an Overdetermined Basis
by: Merkulov, Daniil, et al.
Published: (2024)

Sharing Parameter by Conjugation for Knowledge Graph Embeddings in Complex Space
by: Feng, Xincan, et al.
Published: (2024)

CompressKV: Semantic Retrieval Heads Know What Tokens are Not Important Before Generation
by: Lin, Xiaolin, et al.
Published: (2025)

Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models
by: Yao, Kai, et al.
Published: (2024)

Shared Doubt: Zero-shot Cross-Lingual Confidence Estimation for Language Models
by: Kyriakou, Athina, et al.
Published: (2026)

Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models
by: Bandarkar, Lucas, et al.
Published: (2024)

xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction
by: Chang, Chi-Chih, et al.
Published: (2025)

ResidualTransformer: Residual Low-Rank Learning with Weight-Sharing for Transformer Layers
by: Wang, Yiming, et al.
Published: (2023)

Proxy Compression for Language Modeling
by: Zheng, Lin, et al.
Published: (2026)

BAQ: Efficient Bit Allocation Quantization for Large Language Models
by: Zhang, Chao, et al.
Published: (2025)

Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models
by: Lan, Michael, et al.
Published: (2023)

SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression
by: Wang, Xin, et al.
Published: (2024)

Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Models
by: Liu, Xiaoze, et al.
Published: (2026)

CBQ: Cross-Block Quantization for Large Language Models
by: Ding, Xin, et al.
Published: (2023)

SWSC: Shared Weight for Similar Channel in LLM
by: Zeng, Binrui, et al.
Published: (2025)

SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
by: Chen, Guoxuan, et al.
Published: (2024)

On the Compressibility of Quantized Large Language Models
by: Mao, Yu, et al.
Published: (2024)

Adaptive Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization
by: Ji, Yixin, et al.
Published: (2024)

BasisN: Reprogramming-Free RRAM-Based In-Memory-Computing by Basis Combination for Deep Neural Networks
by: Eldebiky, Amro, et al.
Published: (2024)

KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing
by: Yang, Yifei, et al.
Published: (2024)

Investigating Layer Importance in Large Language Models
by: Zhang, Yang, et al.
Published: (2024)

Model Compression and Efficient Inference for Large Language Models: A Survey
by: Wang, Wenxiao, et al.
Published: (2024)

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
by: Lu, Xudong, et al.
Published: (2024)

ELMES: An Automated Framework for Evaluating Large Language Models in Educational Scenarios
by: Wei, Shou'ang, et al.
Published: (2025)

Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models
by: Chen, Yuyan, et al.
Published: (2024)

Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labels
by: Liu, Chaoqun, et al.
Published: (2024)

Shuttle Between the Instructions and the Parameters of Large Language Models
by: Sun, Wangtao, et al.
Published: (2025)

Accelerating Direct Preference Optimization with Prefix Sharing
by: Wang, Franklin, et al.
Published: (2024)

ORCE: Order-Aware Alignment of Verbalized Confidence in Large Language Models
by: Li, Chen, et al.
Published: (2026)

Data Contamination Report from the 2024 CONDA Shared Task
by: Sainz, Oscar, et al.
Published: (2024)

Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
by: Ban, Hao, et al.
Published: (2025)