Imagen de Portada

Brainstacks: Cross-Domain Cognitive Capabilities via Frozen MoE-LoRA Stacks for Continual LLM Learning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Ayyash, Mohammad R. Abu
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learning
por: Yang, Shu, et al.
Publicado: (2024)

Towards Specialized Generalists: A Multi-Task MoE-LoRA Framework for Domain-Specific LLM Adaptation
por: Yang, Yuxin, et al.
Publicado: (2026)

Three-Phase Transformer
por: Ayyash, Mohammad R. Abu
Publicado: (2026)

LoRA-Gen: Specializing Large Language Model via Online LoRA Generation
por: Xiao, Yicheng, et al.
Publicado: (2025)

Improving LoRA with Variational Learning
por: Cong, Bai, et al.
Publicado: (2025)

Hallucinations and Truth: A Comprehensive Accuracy Evaluation of RAG, LoRA and DoRA
por: Baqar, Mohammad, et al.
Publicado: (2025)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
por: Deng, Guanzhi, et al.
Publicado: (2026)

LoRA-Squeeze: Simple and Effective Post-Tuning and In-Tuning Compression of LoRA Modules
por: Vulić, Ivan, et al.
Publicado: (2026)

LD-MoLE: Learnable Dynamic Routing for Mixture of LoRA Experts
por: Zhuang, Yuan, et al.
Publicado: (2025)

LoRA on the Go: Instance-level Dynamic LoRA Selection and Merging
por: Lee, Seungeon, et al.
Publicado: (2025)

LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
por: Huang, Chengsong, et al.
Publicado: (2023)

Doc-to-LoRA: Learning to Instantly Internalize Contexts
por: Charakorn, Rujikorn, et al.
Publicado: (2026)

HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning
por: Tian, Chunlin, et al.
Publicado: (2024)

LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization
por: Yen, Jui-Nan, et al.
Publicado: (2024)

A Note on LoRA
por: Fomenko, Vlad, et al.
Publicado: (2024)

LoRA Learns Less and Forgets Less
por: Biderman, Dan, et al.
Publicado: (2024)

SAMoRA: Semantic-Aware Mixture of LoRA Experts for Task-Adaptive Learning
por: Shi, Boyan, et al.
Publicado: (2026)

AT-MoE: Adaptive Task-planning Mixture of Experts via LoRA Approach
por: Li, Xurui, et al.
Publicado: (2024)

LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models
por: Ran, Delong, et al.
Publicado: (2025)

KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillation
por: Azimi, Rambod, et al.
Publicado: (2024)

Tina: Tiny Reasoning Models via LoRA
por: Wang, Shangshang, et al.
Publicado: (2025)

Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs
por: Bossy, Thierry, et al.
Publicado: (2025)

LoraMap: Harnessing the Power of LoRA Connections
por: Park, Hyeryun, et al.
Publicado: (2024)

Higher Layers Need More LoRA Experts
por: Gao, Chongyang, et al.
Publicado: (2024)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
por: Manzoni, Andrea
Publicado: (2026)

Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning
por: Lee, Yu-Ang, et al.
Publicado: (2026)

In-Context Meta LoRA Generation
por: Shao, Yihua, et al.
Publicado: (2025)

Align, Don't Divide: Revisiting the LoRA Architecture in Multi-Task Learning
por: Liu, Jinda, et al.
Publicado: (2025)

HypeLoRA: Hyper-Network-Generated LoRA Adapters for Calibrated Language Model Fine-Tuning
por: Trojan, Bartosz, et al.
Publicado: (2026)

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
por: Li, Dengchun, et al.
Publicado: (2024)

DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanism
por: Li, Dengchun, et al.
Publicado: (2025)

Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE
por: Chen, Zeren, et al.
Publicado: (2023)

LoRA Meets Dropout under a Unified Framework
por: Wang, Sheng, et al.
Publicado: (2024)

The Impact of Initialization on LoRA Finetuning Dynamics
por: Hayou, Soufiane, et al.
Publicado: (2024)

LoRA Is Slower Than You Think
por: Ko, Seokmin
Publicado: (2025)

DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Models
por: Zhang, Yuxuan, et al.
Publicado: (2024)

AutoRAG-LoRA: Hallucination-Triggered Knowledge Retuning via Lightweight Adapters
por: Dwivedi, Kaushik, et al.
Publicado: (2025)

Chinese Essay Rhetoric Recognition Using LoRA, In-context Learning and Model Ensemble
por: Lai, Yuxuan, et al.
Publicado: (2026)

Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation
por: Tang, Haozhan, et al.
Publicado: (2026)

LoRA Provides Differential Privacy by Design via Random Sketching
por: Malekmohammadi, Saber, et al.
Publicado: (2024)