Cover Image

SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Han, Jiayi, Du, Liang, Du, Hongwei, Zhou, Xiangguo, Wu, Yiwen, Zheng, Weibo, Han, Donghong
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

AdaFV: Rethinking of Visual-Language alignment for VLM acceleration
by: Han, Jiayi, et al.
Published: (2025)

LoRA Learns Less and Forgets Less
by: Biderman, Dan, et al.
Published: (2024)

Mixture of LoRA Experts
by: Wu, Xun, et al.
Published: (2024)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

TAS-LoRA: Transformer Architecture Search with Mixture-of-LoRA Experts
by: Jeon, Jeimin, et al.
Published: (2026)

ID-LoRA: Identity-Driven Audio-Video Personalization with In-Context LoRA
by: Dahan, Aviad, et al.
Published: (2026)

MOLM: Mixture of LoRA Markers
by: Fares, Samar, et al.
Published: (2025)

Recover-to-Forget: Gradient Reconstruction from LoRA for Efficient LLM Unlearning
by: Liu, Yezi, et al.
Published: (2025)

UnGuide: Learning to Forget with LoRA-Guided Diffusion Models
by: Polowczyk, Agnieszka, et al.
Published: (2025)

Vision as LoRA
by: Wang, Han, et al.
Published: (2025)

LoRA-Drop: Temporal LoRA Decoding for Efficient LLM Inference
by: Rajabzadeh, Hossein, et al.
Published: (2026)

A Note on LoRA
by: Fomenko, Vlad, et al.
Published: (2024)

PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA
by: Wang, Sheng, et al.
Published: (2024)

LoRA-Key: User-Centric LoRA Watermarking for Text-to-Image Diffusion Models
by: Wang, Yaopeng, et al.
Published: (2026)

S-LoRA: Serving Thousands of Concurrent LoRA Adapters
by: Sheng, Ying, et al.
Published: (2023)

FiLoRA: Focus-and-Ignore LoRA for Controllable Feature Reliance
by: Chung, Hyunsuk, et al.
Published: (2026)

Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning
by: Zhao, Ziyu, et al.
Published: (2024)

R-LoRA: Randomized Multi-Head LoRA for Efficient Multi-Task Learning
by: Liu, Jinda, et al.
Published: (2025)

LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks
by: Wang, Hanqing, et al.
Published: (2024)

Lamer-SSL: Layer-aware Mixture of LoRA Experts for Continual Multilingual Expansion of Self-supervised Models without Forgetting
by: Xu, Jing, et al.
Published: (2026)

LoRA Unlearns More and Retains More (Student Abstract)
by: Mittal, Atharv
Published: (2024)

pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning
by: Yi, Liping, et al.
Published: (2023)

Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing
by: Cheng, Ming, et al.
Published: (2025)

A Novel Trustworthy Video Summarization Algorithm Through a Mixture of LoRA Experts
by: Du, Wenzhuo, et al.
Published: (2025)

SAMoRA: Semantic-Aware Mixture of LoRA Experts for Task-Adaptive Learning
by: Shi, Boyan, et al.
Published: (2026)

Higher Layers Need More LoRA Experts
by: Gao, Chongyang, et al.
Published: (2024)

In-Context LoRA for Diffusion Transformers
by: Huang, Lianghua, et al.
Published: (2024)

Multimodal Instruction Tuning with Conditional Mixture of LoRA
by: Shen, Ying, et al.
Published: (2024)

MaLoRA: Gated Modality LoRA for Key-Space Alignment in Multimodal LLM Fine-Tuning
by: Zheng, Xinhan, et al.
Published: (2025)

Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
by: Fan, Chenghao, et al.
Published: (2025)

NP-LoRA: Null Space Projection for Subject-Style LoRA Fusion
by: Chen, Chuheng, et al.
Published: (2025)

Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving
by: Zhang, Yuchen, et al.
Published: (2025)

LoRA Diffusion: Zero-Shot LoRA Synthesis for Diffusion Model Personalization
by: Smith, Ethan, et al.
Published: (2024)

LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluation
by: Zhou, Hongyun, et al.
Published: (2024)

Run LoRA Run: Faster and Lighter LoRA Implementations
by: Cherniuk, Daria, et al.
Published: (2023)

Dual LoRA: Enhancing LoRA with Magnitude and Direction Updates
by: Xu, Yixing, et al.
Published: (2025)

MoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning
by: Ma, Yufei, et al.
Published: (2024)

Understanding Catastrophic Forgetting In LoRA via Mean-Field Attention Dynamics
by: Koubbi, Hugo, et al.
Published: (2024)

Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting
by: Xu, Runze, et al.
Published: (2026)

SCALE-LoRA: Auditing Post-Retrieval LoRA Composition with Residual Merging and View Reliability
by: Zhou, Shuaipeng, et al.
Published: (2026)