Cover Image

GuiLoMo: Allocating Expert Number and Rank for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectors

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Hengyuan, Chen, Xinrong, Qiu, Yingmin, Liang, Xiao, Li, Ziyue, Wang, Guanyu, Li, Weiping, Mo, Tong, So, Hayden Kwok-Hay, Wong, Ngai
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

AT-MoE: Adaptive Task-planning Mixture of Experts via LoRA Approach
by: Li, Xurui, et al.
Published: (2024)

A Sensitivity-Driven Expert Allocation Method in LoRA-MoE for Efficient Fine-Tuning
by: Xu, Junzhou, et al.
Published: (2025)

LoRALib: A Standardized Benchmark for Evaluating LoRA-MoE Methods
by: Wang, Shaoheng, et al.
Published: (2025)

MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learning
by: Yang, Shu, et al.
Published: (2024)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
by: Manzoni, Andrea
Published: (2026)

Hierarchical LoRA MoE for Efficient CTR Model Scaling
by: Zeng, Zhichen, et al.
Published: (2025)

Adaptive and Fine-grained Module-wise Expert Pruning for Efficient LoRA-MoE Fine-Tuning
by: Li, Weihang, et al.
Published: (2026)

TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
by: Kunwar, Pradip, et al.
Published: (2025)

Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE
by: Chen, Zeren, et al.
Published: (2023)

HMVLM: Human Motion-Vision-Lanuage Model via MoE LoRA
by: Hu, Lei, et al.
Published: (2025)

LD-MoLE: Learnable Dynamic Routing for Mixture of LoRA Experts
by: Zhuang, Yuan, et al.
Published: (2025)

MoRA: LoRA Guided Multi-Modal Disease Diagnosis with Missing Modality
by: Shi, Zhiyi, et al.
Published: (2024)

Mixture of LoRA Experts
by: Wu, Xun, et al.
Published: (2024)

TAS-LoRA: Transformer Architecture Search with Mixture-of-LoRA Experts
by: Jeon, Jeimin, et al.
Published: (2026)

LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing
by: Li, Wenbing, et al.
Published: (2025)

A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASR
by: Zheng, Yuang, et al.
Published: (2026)

Noise-Robustness Through Noise: A Framework combining Asymmetric LoRA with Poisoning MoE
by: Wang, Zhaokun, et al.
Published: (2025)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning
by: Wang, Lei, et al.
Published: (2025)

Dual LoRA: Enhancing LoRA with Magnitude and Direction Updates
by: Xu, Yixing, et al.
Published: (2025)

S-LoRA: Serving Thousands of Concurrent LoRA Adapters
by: Sheng, Ying, et al.
Published: (2023)

Bilevel Layer-Positioning LoRA for Real Image Dehazing
by: Zhang, Yan, et al.
Published: (2026)

DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanism
by: Li, Dengchun, et al.
Published: (2025)

Brainstacks: Cross-Domain Cognitive Capabilities via Frozen MoE-LoRA Stacks for Continual LLM Learning
by: Ayyash, Mohammad R. Abu
Published: (2026)

Towards Specialized Generalists: A Multi-Task MoE-LoRA Framework for Domain-Specific LLM Adaptation
by: Yang, Yuxin, et al.
Published: (2026)

MoLEx: Mixture of LoRA Experts in Speech Self-Supervised Models for Audio Deepfake Detection
by: Pan, Zihan, et al.
Published: (2025)

CoMoL: Efficient Mixture of LoRA Experts via Dynamic Core Space Merging
by: Cao, Jie, et al.
Published: (2026)

SHE-LoRA: Selective Homomorphic Encryption for Federated Tuning with Heterogeneous LoRA
by: Liu, Jianmin, et al.
Published: (2025)

Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuning
by: Gou, Yunhao, et al.
Published: (2023)

Residual Decoding: Mitigating Hallucinations in Large Vision-Language Models via History-Aware Residual Guidance
by: Chen, Xinrong, et al.
Published: (2026)

Vision as LoRA
by: Wang, Han, et al.
Published: (2025)

Uni-LoRA: One Vector is All You Need
by: Li, Kaiyang, et al.
Published: (2025)

AlphaLoRA: Assigning LoRA Experts Based on Layer Training Quality
by: Qing, Peijun, et al.
Published: (2024)

FIM-LoRA: Task-Informative Rank Allocation for LoRA via Calibration-Time Gradient-Variance Estimation
by: Sathyavageeswaran, Ramakrishnan
Published: (2026)

Run LoRA Run: Faster and Lighter LoRA Implementations
by: Cherniuk, Daria, et al.
Published: (2023)

Post-Optimization Adaptive Rank Allocation for LoRA
by: Kumaravelu, Vishnuprasadh, et al.
Published: (2026)

Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
by: Fan, Chenghao, et al.
Published: (2025)

Cross-LoRA: A Data-Free LoRA Transfer Framework across Heterogeneous LLMs
by: Xia, Feifan, et al.
Published: (2025)

HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning
by: Tian, Chunlin, et al.
Published: (2024)

Ortho-Hydra: Orthogonalized Experts for DiT LoRA
by: Ji, Seunghyun
Published: (2026)