Image de couverture de livre

LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Li, Guangyan, Tang, Yongqiang, Zhang, Wensheng
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

MGAA: Multi-Granular Adaptive Allocation fof Low-Rank Compression of LLMs
par: Li, Guangyan, et autres
Publié: (2025)

RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models
par: Hao, Haoran, et autres
Publié: (2024)

LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades
par: Li, Yanan, et autres
Publié: (2025)

DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Models
par: Zhang, Yuxuan, et autres
Publié: (2024)

On the Compressibility of Quantized Large Language Models
par: Mao, Yu, et autres
Publié: (2024)

LoLCATs: On Low-Rank Linearizing of Large Language Models
par: Zhang, Michael, et autres
Publié: (2024)

Structured Agent Distillation for Large Language Model
par: Liu, Jun, et autres
Publié: (2025)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
par: Zhang, Jun, et autres
Publié: (2025)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
par: Chen, Yukang, et autres
Publié: (2023)

Federated Learning with Layer Skipping: Efficient Training of Large Language Models for Healthcare NLP
par: Zhang, Lihong, et autres
Publié: (2025)

A Survey on LoRA of Large Language Models
par: Mao, Yuren, et autres
Publié: (2024)

Depth-Recurrent Attention Mixtures: Giving Latent Reasoning the Attention it Deserves
par: Knupp, Jonas, et autres
Publié: (2026)

Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation
par: Zhang, Hongxiang, et autres
Publié: (2025)

On the Geometric Structure of Layer Updates in Deep Language Models
par: Yoo, Jun-Sik
Publié: (2026)

LayerIF: Estimating Layer Quality for Large Language Models using Influence Functions
par: Askari, Hadi, et autres
Publié: (2025)

Clustering-driven Memory Compression for On-device Large Language Models
par: Bohdal, Ondrej, et autres
Publié: (2026)

In-context Autoencoder for Context Compression in a Large Language Model
par: Ge, Tao, et autres
Publié: (2023)

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit
par: Gong, Ruihao, et autres
Publié: (2024)

Adaptive Pruning for Large Language Models with Structural Importance Awareness
par: Zheng, Haotian, et autres
Publié: (2024)

Mixtures of SubExperts for Large Language Continual Learning
par: Kang, Haeyong
Publié: (2025)

SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
par: Chen, Guoxuan, et autres
Publié: (2024)

JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models
par: Dragomir, Alexandra, et autres
Publié: (2026)

SEE: Sememe Entanglement Encoding for Transformer-bases Models Compression
par: Zhang, Jing, et autres
Publié: (2024)

Model Compression and Efficient Inference for Large Language Models: A Survey
par: Wang, Wenxiao, et autres
Publié: (2024)

RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agents
par: Kagaya, Tomoyuki, et autres
Publié: (2024)

Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning
par: Liu, Z, et autres
Publié: (2024)

Sample-aware Adaptive Structured Pruning for Large Language Models
par: Kong, Jun, et autres
Publié: (2025)

MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
par: Liu, Akide, et autres
Publié: (2024)

Measuring Social Norms of Large Language Models
par: Yuan, Ye, et autres
Publié: (2024)

Large Language Models to Diffusion Finetuning
par: Cetin, Edoardo, et autres
Publié: (2025)

Align to Structure: Aligning Large Language Models with Structural Information
par: Kim, Zae Myung, et autres
Publié: (2025)

SLOT: Structuring the Output of Large Language Models
par: Wang, Darren Yow-Bang, et autres
Publié: (2025)

LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models
par: Elesedy, Hayder, et autres
Publié: (2024)

Dynamic Universal Approximation Theory: The Basic Theory for Transformer-based Large Language Models
par: Wang, Wei, et autres
Publié: (2024)

Prompting Fairness: Integrating Causality to Debias Large Language Models
par: Li, Jingling, et autres
Publié: (2024)

FineZip : Pushing the Limits of Large Language Models for Practical Lossless Text Compression
par: Mittu, Fazal, et autres
Publié: (2024)

DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
par: Chuang, Yung-Sung, et autres
Publié: (2023)

Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models
par: Bandarkar, Lucas, et autres
Publié: (2024)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
par: Yang, Yifan, et autres
Publié: (2024)

Layer by Layer: Uncovering Hidden Representations in Language Models
par: Skean, Oscar, et autres
Publié: (2025)