Cover Image

Mixture-of-Experts with Gradient Conflict-Driven Subspace Topology Pruning for Emergent Modularity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gan, Yuxing, Lei, Ziyu
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Grassmannian Mixture-of-Experts: Concentration-Controlled Routing on Subspace Manifolds
by: Shihab, Ibne Farabi, et al.
Published: (2026)

Mixture of Length and Pruning Experts for Knowledge Graphs Reasoning
by: Du, Enjun, et al.
Published: (2025)

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
by: Li, Lujun, et al.
Published: (2025)

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
by: Zhang, Zeliang, et al.
Published: (2024)

A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts
by: Chowdhury, Mohammed Nowaz Rabbani, et al.
Published: (2024)

Mosaic Pruning: A Hierarchical Framework for Generalizable Pruning of Mixture-of-Experts Models
by: Hu, Wentao, et al.
Published: (2025)

Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning
by: Zhao, Ziyu, et al.
Published: (2024)

Multi-Subspace Multi-Modal Modeling for Diffusion Models: Estimation, Convergence and Mixture of Experts
by: Yang, Ruofeng, et al.
Published: (2026)

Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs
by: Liu, Enshu, et al.
Published: (2024)

Train Separately, Merge Together: Modular Post-Training with Mixture-of-Experts
by: Morrison, Jacob, et al.
Published: (2026)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
by: Kim, Junhyuck, et al.
Published: (2026)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
by: Lu, Xudong, et al.
Published: (2024)

Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations
by: Dong, Zican, et al.
Published: (2025)

Mixture of Experts (MoE): A Big Data Perspective
by: Gan, Wensheng, et al.
Published: (2025)

Subspace Node Pruning
by: Offergeld, Joshua, et al.
Published: (2024)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
by: Yang, Cheng, et al.
Published: (2024)

$μ$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
by: Koike-Akino, Toshiaki, et al.
Published: (2025)

Mixture of Experts Provably Detect and Learn the Latent Cluster Structure in Gradient-Based Learning
by: Kawata, Ryotaro, et al.
Published: (2025)

ResPrune: Text-Conditioned Subspace Reconstruction for Visual Token Pruning in Large Vision-Language Models
by: Li, Xu, et al.
Published: (2026)

Mixture-of-Subspaces in Low-Rank Adaptation
by: Wu, Taiqiang, et al.
Published: (2024)

Uncertainty Makes It Stable: Curiosity-Driven Quantized Mixture-of-Experts
by: Ordóñez, Sebastián Andrés Cajas, et al.
Published: (2025)

Learning More Generalized Experts by Merging Experts in Mixture-of-Experts
by: Park, Sejik
Published: (2024)

Matched Topological Subspace Detector
by: Liu, Chengen, et al.
Published: (2025)

GraphMoRE: Mitigating Topological Heterogeneity via Mixture of Riemannian Experts
by: Guo, Zihao, et al.
Published: (2024)

Disentangling Task Conflicts in Multi-Task LoRA via Orthogonal Gradient Projection
by: Yang, Ziyu, et al.
Published: (2026)

PrunedLoRA: Robust Gradient-Based structured pruning for Low-rank Adaptation in Fine-tuning
by: Yu, Xin, et al.
Published: (2025)

Utility-Driven Speculative Decoding for Mixture-of-Experts
by: Saxena, Anish, et al.
Published: (2025)

Identifying Policy Gradient Subspaces
by: Schneider, Jan, et al.
Published: (2024)

Expert Merging in Sparse Mixture of Experts with Nash Bargaining
by: Nguyen, Dung V., et al.
Published: (2025)

$μ$-Parametrization for Mixture of Experts
by: Małaśnicki, Jan, et al.
Published: (2025)

Path-Constrained Mixture-of-Experts
by: Gu, Zijin, et al.
Published: (2026)

Expert Race: A Flexible Routing Strategy for Scaling Diffusion Transformer with Mixture of Experts
by: Yuan, Yike, et al.
Published: (2025)

ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning
by: Gao, Shangqian, et al.
Published: (2025)

Efficient Estimation of the Central Mean Subspace via Smoothed Gradient Outer Products
by: Yuan, Gan, et al.
Published: (2023)

Topology-Aware Multiscale Mixture of Experts for Efficient Molecular Property Prediction
by: Nguyen, Long D., et al.
Published: (2026)

Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning
by: Zhao, Ziyu, et al.
Published: (2025)

Mixture of Raytraced Experts
by: Perin, Andrea, et al.
Published: (2025)

Mixture of Lookup Experts
by: Jie, Shibo, et al.
Published: (2025)

PMODE: Theoretically Grounded and Modular Mixture Modeling
by: Vandermeulen, Robert A.
Published: (2025)

Mixture of Experts in a Mixture of RL settings
by: Willi, Timon, et al.
Published: (2024)