Imagen de Portada

Low-Rank Adapters Meet Neural Architecture Search for LLM Compression

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Muñoz, J. Pablo, Yuan, Jinjie, Jain, Nilesh
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Shears: Unstructured Sparsity with Neural Low-rank Adapter Search
por: Muñoz, J. Pablo, et al.
Publicado: (2024)

SQFT: Low-cost Model Adaptation in Low-precision Sparse Foundation Models
por: Muñoz, Juan Pablo, et al.
Publicado: (2024)

Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models
por: Muñoz, J. Pablo, et al.
Publicado: (2025)

Ensembles of Low-Rank Expert Adapters
por: Li, Yinghao, et al.
Publicado: (2025)

MultiPruner: Balanced Structure Removal in Foundation Models
por: Muñoz, J. Pablo, et al.
Publicado: (2025)

Multiple Choice Learning of Low-Rank Adapters for Language Modeling
por: Letzelter, Victor, et al.
Publicado: (2025)

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?
por: Wang, Zhengbo, et al.
Publicado: (2024)

zFLoRA: Zero-Latency Fused Low-Rank Adapters
por: Gowda, Dhananjaya, et al.
Publicado: (2025)

Training-Free Bayesianization for Low-Rank Adapters of Large Language Models
por: Shi, Haizhou, et al.
Publicado: (2024)

Learning Adapter Rank via Symmetry Breaking
por: Doyle, Cooper, et al.
Publicado: (2025)

Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters
por: Wang, Xinjue, et al.
Publicado: (2026)

OrchMoE: Efficient Multi-Adapter Learning with Task-Skill Synergy
por: Wang, Haowen, et al.
Publicado: (2024)

Eigen Attention: Attention in Low-Rank Space for KV Cache Compression
por: Saxena, Utkarsh, et al.
Publicado: (2024)

OjaKV: Context-Aware Online Low-Rank KV Cache Compression
por: Zhu, Yuxuan, et al.
Publicado: (2025)

MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures
por: Ni, Jinjie, et al.
Publicado: (2024)

KVCrush: Key value cache size-reduction using similarity in head-behaviour
por: Jha, Gopi Krishna, et al.
Publicado: (2025)

LoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuning
por: Perin, Gabriel J., et al.
Publicado: (2025)

GoRA: Gradient-driven Adaptive Low Rank Adaptation
por: He, Haonan, et al.
Publicado: (2025)

Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search
por: Gu, Yuxian, et al.
Publicado: (2025)

Personas within Parameters: Fine-Tuning Small Language Models with Low-Rank Adapters to Mimic User Behaviors
por: Thakur, Himanshu, et al.
Publicado: (2025)

LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy
por: Zhang, Rongzhi, et al.
Publicado: (2024)

EliteKV: Scalable KV Cache Compression via RoPE Frequency Selection and Joint Low-Rank Projection
por: Zhou, Yuhao, et al.
Publicado: (2025)

Quantization Dominates Rank Reduction for KV-Cache Compression
por: Salfati, Samuel
Publicado: (2026)

ChameleonLLM: Batch-Aware Dynamic Low-Rank Adaptation via Inference-Time Clusters
por: Yuksel, Kamer Ali, et al.
Publicado: (2025)

MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning
por: Modoranu, Ionut-Vlad, et al.
Publicado: (2026)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
por: Tang, Chuanyu, et al.
Publicado: (2024)

JuStRank: Benchmarking LLM Judges for System Ranking
por: Gera, Ariel, et al.
Publicado: (2024)

CharPoet: A Chinese Classical Poetry Generation System Based on Token-free LLM
por: Yu, Chengyue, et al.
Publicado: (2024)

The Expressive Power of Low-Rank Adaptation
por: Zeng, Yuchen, et al.
Publicado: (2023)

A Transformer-based Neural Architecture Search Method
por: Wang, Shang, et al.
Publicado: (2025)

CPRM: A LLM-based Continual Pre-training Framework for Relevance Modeling in Commercial Search
por: Wu, Kaixin, et al.
Publicado: (2024)

LiteSearch: Efficacious Tree Search for LLM
por: Wang, Ante, et al.
Publicado: (2024)

Solo Connection: A Parameter Efficient Fine-Tuning Technique for Transformers
por: Pathak, Harsh Nilesh, et al.
Publicado: (2025)

Low-Rank Quantization-Aware Training for LLMs
por: Bondarenko, Yelysei, et al.
Publicado: (2024)

Batched Low-Rank Adaptation of Foundation Models
por: Wen, Yeming, et al.
Publicado: (2023)

Variational Low-Rank Adaptation Using IVON
por: Cong, Bai, et al.
Publicado: (2024)

Training Acceleration of Low-Rank Decomposed Networks using Sequential Freezing and Rank Quantization
por: Hajimolahoseini, Habib, et al.
Publicado: (2023)

Rethinking the Unsolvable: When In-Context Search Meets Test-Time Scaling
por: Xia, Fanzeng, et al.
Publicado: (2025)

Not All Adapters Matter: Selective Adapter Freezing for Memory-Efficient Fine-Tuning of Language Models
por: Son, Hyegang, et al.
Publicado: (2024)

SwitchLoRA: Switched Low-Rank Adaptation Can Learn Full-Rank Information
por: Zhou, Kaiye, et al.
Publicado: (2024)