Image de couverture de livre

MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yang, Shu, Ali, Muhammad Asif, Wang, Cheng-Long, Hu, Lijie, Wang, Di
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
par: Manzoni, Andrea
Publié: (2026)

Brainstacks: Cross-Domain Cognitive Capabilities via Frozen MoE-LoRA Stacks for Continual LLM Learning
par: Ayyash, Mohammad R. Abu
Publié: (2026)

Towards Specialized Generalists: A Multi-Task MoE-LoRA Framework for Domain-Specific LLM Adaptation
par: Yang, Yuxin, et autres
Publié: (2026)

Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE
par: Chen, Zeren, et autres
Publié: (2023)

Hierarchical LoRA MoE for Efficient CTR Model Scaling
par: Zeng, Zhichen, et autres
Publié: (2025)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
par: Deng, Guanzhi, et autres
Publié: (2026)

ELDER: Enhancing Lifelong Model Editing with Mixture-of-LoRA
par: Li, Jiaang, et autres
Publié: (2024)

LoRALib: A Standardized Benchmark for Evaluating LoRA-MoE Methods
par: Wang, Shaoheng, et autres
Publié: (2025)

GuiLoMo: Allocating Expert Number and Rank for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectors
par: Zhang, Hengyuan, et autres
Publié: (2025)

A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASR
par: Zheng, Yuang, et autres
Publié: (2026)

LD-MoLE: Learnable Dynamic Routing for Mixture of LoRA Experts
par: Zhuang, Yuan, et autres
Publié: (2025)

LoRA-Gen: Specializing Large Language Model via Online LoRA Generation
par: Xiao, Yicheng, et autres
Publié: (2025)

HMVLM: Human Motion-Vision-Lanuage Model via MoE LoRA
par: Hu, Lei, et autres
Publié: (2025)

LLaDA-MoE: A Sparse MoE Diffusion Language Model
par: Zhu, Fengqi, et autres
Publié: (2025)

Dialectical Alignment: Resolving the Tension of 3H and Security Threats of LLMs
par: Yang, Shu, et autres
Publié: (2024)

AT-MoE: Adaptive Task-planning Mixture of Experts via LoRA Approach
par: Li, Xurui, et autres
Publié: (2024)

Noise-Robustness Through Noise: A Framework combining Asymmetric LoRA with Poisoning MoE
par: Wang, Zhaokun, et autres
Publié: (2025)

Sigma-MoE-Tiny Technical Report
par: Hu, Qingguo, et autres
Publié: (2025)

DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanism
par: Li, Dengchun, et autres
Publié: (2025)

GRIN: GRadient-INformed MoE
par: Liu, Liyuan, et autres
Publié: (2024)

Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs
par: Bossy, Thierry, et autres
Publié: (2025)

LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models
par: Ran, Delong, et autres
Publié: (2025)

ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs
par: Chen, Xunlei, et autres
Publié: (2026)

A Sensitivity-Driven Expert Allocation Method in LoRA-MoE for Efficient Fine-Tuning
par: Xu, Junzhou, et autres
Publié: (2025)

TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
par: Kunwar, Pradip, et autres
Publié: (2025)

In-Context Meta LoRA Generation
par: Shao, Yihua, et autres
Publié: (2025)

LoRA on the Go: Instance-level Dynamic LoRA Selection and Merging
par: Lee, Seungeon, et autres
Publié: (2025)

Improving LoRA with Variational Learning
par: Cong, Bai, et autres
Publié: (2025)

LoRA-Squeeze: Simple and Effective Post-Tuning and In-Tuning Compression of LoRA Modules
par: Vulić, Ivan, et autres
Publié: (2026)

Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts
par: Li, Yunxin, et autres
Publié: (2024)

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
par: Li, Dengchun, et autres
Publié: (2024)

SAMoRA: Semantic-Aware Mixture of LoRA Experts for Task-Adaptive Learning
par: Shi, Boyan, et autres
Publié: (2026)

LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization
par: Yen, Jui-Nan, et autres
Publié: (2024)

Doc-to-LoRA: Learning to Instantly Internalize Contexts
par: Charakorn, Rujikorn, et autres
Publié: (2026)

A Note on LoRA
par: Fomenko, Vlad, et autres
Publié: (2024)

LoRA Learns Less and Forgets Less
par: Biderman, Dan, et autres
Publié: (2024)

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
par: Shi, Jingze, et autres
Publié: (2026)

Align, Don't Divide: Revisiting the LoRA Architecture in Multi-Task Learning
par: Liu, Jinda, et autres
Publié: (2025)

HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning
par: Tian, Chunlin, et autres
Publié: (2024)

Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
par: Li, Bo, et autres
Publié: (2026)