Imagen de Portada

Towards Modular LLMs by Building and Reusing a Library of LoRAs

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Ostapenko, Oleksiy, Su, Zhan, Ponti, Edoardo Maria, Charlin, Laurent, Roux, Nicolas Le, Pereira, Matheus, Caccia, Lucas, Sordoni, Alessandro
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Guiding Language Model Reasoning with Planning Tokens
por: Wang, Xinyi, et al.
Publicado: (2023)

Training Plug-n-Play Knowledge Modules with Deep Context Distillation
por: Caccia, Lucas, et al.
Publicado: (2025)

Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
por: Arnob, Samin Yeasar, et al.
Publicado: (2025)

Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering
por: Zhao, Ziyu, et al.
Publicado: (2024)

LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks
por: Prabhakar, Akshara, et al.
Publicado: (2024)

Improving Recursive Transformers with Mixture of LoRAs
por: Nouriborji, Mohammadmahdi, et al.
Publicado: (2025)

Learning to Extract Context for Context-Aware LLM Inference
por: Kim, Minseon, et al.
Publicado: (2025)

Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing
por: Cheng, Ming, et al.
Publicado: (2025)

Improving Context-Aware Preference Modeling for Language Models
por: Pitis, Silviu, et al.
Publicado: (2024)

Trade-offs in Ensembling, Merging and Routing Among Parameter-Efficient Experts
por: Lotfi, Sanae, et al.
Publicado: (2026)

The Appeal and Reality of Recycling LoRAs with Adaptive Merging
por: Liu, Haokun, et al.
Publicado: (2026)

LoRAGuard: An Effective Black-box Watermarking Approach for LoRAs
por: Lv, Peizhuo, et al.
Publicado: (2025)

K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs
por: Ouyang, Ziheng, et al.
Publicado: (2025)

qa-FLoRA: Data-free query-adaptive Fusion of LoRAs for LLMs
por: Shukla, Shreya, et al.
Publicado: (2025)

Two Is Better Than One: Rotations Scale LoRAs
por: Guo, Hongcan, et al.
Publicado: (2025)

Rank-1 LoRAs Encode Interpretable Reasoning Signals
por: Ward, Jake, et al.
Publicado: (2025)

Subject or Style: Adaptive and Training-Free Mixture of LoRAs
por: Zhang, Jia-Chen, et al.
Publicado: (2025)

Dynamic Training-Free Fusion of Subject and Style LoRAs
por: Cao, Qinglong, et al.
Publicado: (2026)

Separating Shared and Domain-Specific LoRAs for Multi-Domain Learning
por: Takama, Yusaku, et al.
Publicado: (2025)

Spanning the Visual Analogy Space with a Weight Basis of LoRAs
por: Manor, Hila, et al.
Publicado: (2026)

ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning
por: Qiu, Ruizhong, et al.
Publicado: (2026)

Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models
por: Feng, Wenfeng, et al.
Publicado: (2024)

Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
por: Ban, Hao, et al.
Publicado: (2025)

CARLoS: Retrieval via Concise Assessment Representation of LoRAs at Scale
por: Sarfaty, Shahar, et al.
Publicado: (2025)

Learning Attentional Mixture of LoRAs for Language Model Continual Learning
por: Liu, Jialin, et al.
Publicado: (2024)

ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs
por: Shah, Viraj, et al.
Publicado: (2023)

VinePPO: Refining Credit Assignment in RL Training of LLMs
por: Kazemnejad, Amirhossein, et al.
Publicado: (2024)

Learning to Solve Complex Problems via Dataset Decomposition
por: Zhao, Wanru, et al.
Publicado: (2026)

Privileged Information Distillation for Language Models
por: Penaloza, Emiliano, et al.
Publicado: (2026)

Position: Pause Recycling LoRAs and Prioritize Mechanisms to Uncover Limits and Effectiveness
por: Chen, Mei-Yen, et al.
Publicado: (2025)

ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion
por: Khan, Rana Muhammad Shahroz, et al.
Publicado: (2025)

ORACLE: Leveraging Mutual Information for Consistent Character Generation with LoRAs in Diffusion Models
por: Akdemir, Kiymet, et al.
Publicado: (2024)

LoRA.rar: Learning to Merge LoRAs via Hypernetworks for Subject-Style Conditioned Image Generation
por: Shenaj, Donald, et al.
Publicado: (2024)

MoSLD: An Extremely Parameter-Efficient Mixture-of-Shared LoRAs for Multi-Task Learning
por: Zhao, Lulu, et al.
Publicado: (2024)

Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models
por: Putterman, Theo, et al.
Publicado: (2024)

HyperDet: Generalizable Detection of Synthesized Images by Generating and Merging A Mixture of Hyper LoRAs
por: Cao, Huangsen, et al.
Publicado: (2024)

LoRAFusion: Efficient LoRA Fine-Tuning for LLMs
por: Zhu, Zhanda, et al.
Publicado: (2025)

BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills
por: Sonwane, Atharv, et al.
Publicado: (2025)

A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
por: Corbeil, Jean-Philippe, et al.
Publicado: (2025)

Modular Deep Learning
por: Pfeiffer, Jonas, et al.
Publicado: (2023)