Imagen de Portada

MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptation

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yuan, Shen, Zheng, Yin, Wang, Taifeng, Liu, Binbin, Xu, Hongteng
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models
por: Xu, Zukang, et al.
Publicado: (2026)

AIMER: Calibration-Free Task-Agnostic MoE Pruning
por: Liu, Zongfang, et al.
Publicado: (2026)

DynaMo: Runtime Switchable Quantization for MoE with Cross-Dataset Adaptation
por: Zheng, Zihao, et al.
Publicado: (2025)

Expert Divergence Learning for MoE-based Language Models
por: Li, Jiaang, et al.
Publicado: (2026)

eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inference
por: Tairin, Suraiya, et al.
Publicado: (2025)

Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation
por: Yuan, Shen, et al.
Publicado: (2024)

EvoESAP: Non-Uniform Expert Pruning for Sparse MoE
por: Liu, Zongfang, et al.
Publicado: (2026)

Harder Tasks Need More Experts: Dynamic Routing in MoE Models
por: Huang, Quzhe, et al.
Publicado: (2024)

MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
por: Chen, Xiaodong, et al.
Publicado: (2025)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
por: Ma, Songkai, et al.
Publicado: (2025)

MoE-PHDS: One MoE checkpoint for flexible runtime sparsity
por: Hannah, Lauren. A, et al.
Publicado: (2025)

AdapMoE: Adaptive Sensitivity-based Expert Gating and Management for Efficient MoE Inference
por: Zhong, Shuzhang, et al.
Publicado: (2024)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
por: Cao, Shiyi, et al.
Publicado: (2024)

MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design
por: Duanmu, Haojie, et al.
Publicado: (2025)

GW-MoE: Resolving Uncertainty in MoE Router with Global Workspace Theory
por: Wu, Haoze, et al.
Publicado: (2024)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
por: Xia, Xinfeng, et al.
Publicado: (2025)

GRIN: GRadient-INformed MoE
por: Liu, Liyuan, et al.
Publicado: (2024)

Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoE
por: Huang, Haiduo, et al.
Publicado: (2025)

Hierarchical LoRA MoE for Efficient CTR Model Scaling
por: Zeng, Zhichen, et al.
Publicado: (2025)

Towards Better Multi-head Attention via Channel-wise Sample Permutation
por: Yuan, Shen, et al.
Publicado: (2024)

Grouter: Decoupling Routing from Representation for Accelerated MoE Training
por: Xu, Yuqi, et al.
Publicado: (2026)

SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
por: Tang, Shengkun, et al.
Publicado: (2026)

MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
por: Miao, Ruijie, et al.
Publicado: (2025)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
por: Manzoni, Andrea
Publicado: (2026)

Towards Specialized Generalists: A Multi-Task MoE-LoRA Framework for Domain-Specific LLM Adaptation
por: Yang, Yuxin, et al.
Publicado: (2026)

MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching
por: Xu, Tairan, et al.
Publicado: (2025)

Monkey Jump : MoE-Style PEFT for Efficient Multi-Task Learning
por: Prottasha, Nusrat Jahan, et al.
Publicado: (2026)

Delta Decompression for MoE-based LLMs Compression
por: Gu, Hao, et al.
Publicado: (2025)

MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
por: Xue, Leyang, et al.
Publicado: (2024)

LocMoE: A Low-Overhead MoE for Large Language Model Training
por: Li, Jing, et al.
Publicado: (2024)

Faster MoE LLM Inference for Extremely Large Models
por: Yang, Haoqi, et al.
Publicado: (2025)

MoE-Prefill: Zero Redundancy Overheads in MoE Prefill Serving
por: Su, Zhaoyuan, et al.
Publicado: (2026)

MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core
por: Liu, Dennis, et al.
Publicado: (2025)

MoE-GPS: Guidlines for Prediction Strategy for Dynamic Expert Duplication in MoE Load Balancing
por: Ma, Haiyue, et al.
Publicado: (2025)

DOT-MoE: Differentiable Optimal Transport for MoEfication
por: Bamba, Udbhav, et al.
Publicado: (2026)

FluxMoE: Decoupling Expert Residency for High-Performance MoE Serving
por: Liu, Qingxiu, et al.
Publicado: (2026)

Multi-Head LatentMoE and Head Parallel: Communication-Efficient and Deterministic MoE Parallelism
por: Cui, Chenwei, et al.
Publicado: (2026)

Faster, Smaller, and Smarter: Task-Aware Expert Merging for Online MoE Inference
por: Han, Ziyi, et al.
Publicado: (2025)

MoETTA: Test-Time Adaptation Under Mixed Distribution Shifts with MoE-LayerNorm
por: Fan, Xiao, et al.
Publicado: (2025)

Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
por: Li, Junzhuo, et al.
Publicado: (2025)