Copertina

GW-MoE: Resolving Uncertainty in MoE Router with Global Workspace Theory

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wu, Haoze, Qiu, Zihan, Wang, Zili, Zhao, Hang, Fu, Jie
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
di: Zeng, Zhiyuan, et al.
Pubblicazione: (2024)

Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
di: Ma, Wenhan, et al.
Pubblicazione: (2025)

MoE-PHDS: One MoE checkpoint for flexible runtime sparsity
di: Hannah, Lauren. A, et al.
Pubblicazione: (2025)

Is Retraining-Free Enough? The Necessity of Router Calibration for Efficient MoE Compression
di: Hyeon, Sieun, et al.
Pubblicazione: (2026)

KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models
di: Xu, Zukang, et al.
Pubblicazione: (2026)

Optimizing MoE Routers: Design, Implementation, and Evaluation in Transformer Models
di: Harvey, Daniel Fidel, et al.
Pubblicazione: (2025)

BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization
di: Zhao, Jiayu, et al.
Pubblicazione: (2026)

FFT-MoE: Efficient Federated Fine-Tuning for Foundation Models via Large-scale Sparse MoE under Heterogeneous Edge
di: Hu, Gang, et al.
Pubblicazione: (2025)

HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts
di: Zhao, Hao, et al.
Pubblicazione: (2024)

GRIN: GRadient-INformed MoE
di: Liu, Liyuan, et al.
Pubblicazione: (2024)

Grouter: Decoupling Routing from Representation for Accelerated MoE Training
di: Xu, Yuqi, et al.
Pubblicazione: (2026)

DOT-MoE: Differentiable Optimal Transport for MoEfication
di: Bamba, Udbhav, et al.
Pubblicazione: (2026)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
di: Cao, Shiyi, et al.
Pubblicazione: (2024)

Collaborative Compression for Large-Scale MoE Deployment on Edge
di: Chen, Yixiao, et al.
Pubblicazione: (2025)

SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations
di: Guo, Wentao, et al.
Pubblicazione: (2025)

STAMImputer: Spatio-Temporal Attention MoE for Traffic Data Imputation
di: Wang, Yiming, et al.
Pubblicazione: (2025)

Hierarchical LoRA MoE for Efficient CTR Model Scaling
di: Zeng, Zhichen, et al.
Pubblicazione: (2025)

MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
di: Li, Shuhuai, et al.
Pubblicazione: (2026)

Exploiting the Experts: Unauthorized Compression in MoE-LLMs
di: Neogi, Pinaki Prasad Guha, et al.
Pubblicazione: (2025)

(GG) MoE vs. MLP on Tabular Data
di: Chernov, Andrei
Pubblicazione: (2025)

MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE
di: Huang, Zongle, et al.
Pubblicazione: (2025)

PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning
di: Hou, Zhiyan, et al.
Pubblicazione: (2026)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
di: Xie, Yanyue, et al.
Pubblicazione: (2024)

SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
di: Tang, Shengkun, et al.
Pubblicazione: (2026)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
di: Gu, Naibin, et al.
Pubblicazione: (2025)

Expert Divergence Learning for MoE-based Language Models
di: Li, Jiaang, et al.
Pubblicazione: (2026)

SD-MoE: Spectral Decomposition for Effective Expert Specialization
di: Huang, Ruijun, et al.
Pubblicazione: (2026)

MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design
di: Duanmu, Haojie, et al.
Pubblicazione: (2025)

Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers
di: Lau, Tim Tsz-Kit, et al.
Pubblicazione: (2026)

Innovator: Scientific Continued Pretraining with Fine-grained MoE Upcycling
di: Liao, Ning, et al.
Pubblicazione: (2025)

MoEITS: A Green AI approach for simplifying MoE-LLMs
di: Balderas, Luis, et al.
Pubblicazione: (2026)

Mixture of Experts (MoE): A Big Data Perspective
di: Gan, Wensheng, et al.
Pubblicazione: (2025)

SDG-MoE: Signed Debate Graph Mixture-of-Experts
di: Kulibaba, Stepan, et al.
Pubblicazione: (2026)

Awakening Dormant Experts:Counterfactual Routing to Mitigate MoE Hallucinations
di: Hu, Wentao, et al.
Pubblicazione: (2026)

LocMoE: A Low-Overhead MoE for Large Language Model Training
di: Li, Jing, et al.
Pubblicazione: (2024)

MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference
di: Li, Bo, et al.
Pubblicazione: (2026)

Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts
di: Yun, Sukwon, et al.
Pubblicazione: (2024)

Revisiting MoE and Dense Speed-Accuracy Comparisons for LLM Training
di: Du, Xianzhi, et al.
Pubblicazione: (2024)

Towards an empirical understanding of MoE design choices
di: Fan, Dongyang, et al.
Pubblicazione: (2024)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
di: Jin, Peng, et al.
Pubblicazione: (2024)