Imagen de Portada

Sparsity is Combinatorial Depth: Quantifying MoE Expressivity via Tropical Geometry

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Su, Ye, Tang, Huayi, Gong, Zixuan, Liu, Yong
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Expressivity of Transformers: A Tropical Geometry Perspective
por: Su, Ye, et al.
Publicado: (2026)

Effective Frontiers: A Unification of Neural Scaling Laws
por: Zou, Jiaxuan, et al.
Publicado: (2026)

Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization
por: Gong, Zixuan, et al.
Publicado: (2025)

MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
por: Xue, Leyang, et al.
Publicado: (2024)

Improving MoE Compute Efficiency by Composing Weight and Data Sparsity
por: Kilian, Maciej, et al.
Publicado: (2026)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
por: Xia, Xinfeng, et al.
Publicado: (2025)

Information-Theoretic Generalization Bounds for Transductive Learning and its Applications
por: Tang, Huayi, et al.
Publicado: (2023)

PAC-Bayesian Generalization Bounds for Graph Convolutional Networks on Inductive Node Classification
por: Tang, Huayi, et al.
Publicado: (2025)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
por: Li, Zichong, et al.
Publicado: (2025)

AIMER: Calibration-Free Task-Agnostic MoE Pruning
por: Liu, Zongfang, et al.
Publicado: (2026)

EvoESAP: Non-Uniform Expert Pruning for Sparse MoE
por: Liu, Zongfang, et al.
Publicado: (2026)

MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
por: Miao, Ruijie, et al.
Publicado: (2025)

MoE-PHDS: One MoE checkpoint for flexible runtime sparsity
por: Hannah, Lauren. A, et al.
Publicado: (2025)

Anchor-MoE: A Mean-Anchored Mixture of Experts For Probabilistic Regression
por: Su, Baozhuo, et al.
Publicado: (2025)

DualSparse-MoE: Coordinating Tensor/Neuron-Level Sparsity with Expert Partition and Reconstruction
por: Cai, Weilin, et al.
Publicado: (2025)

RAST-MoE-RL: A Regime-Aware Spatio-Temporal MoE Framework for Deep Reinforcement Learning in Ride-Hailing
por: Tang, Yuhan, et al.
Publicado: (2025)

MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
por: Li, Shuhuai, et al.
Publicado: (2026)

DA-MoE: Addressing Depth-Sensitivity in Graph-Level Analysis through Mixture of Experts
por: Yao, Zelin, et al.
Publicado: (2024)

GW-MoE: Resolving Uncertainty in MoE Router with Global Workspace Theory
por: Wu, Haoze, et al.
Publicado: (2024)

Retraining-Free Merging of Sparse MoE via Hierarchical Clustering
por: Chen, I-Chun, et al.
Publicado: (2024)

Perfect Alignment May be Poisonous to Graph Contrastive Learning
por: Liu, Jingyu, et al.
Publicado: (2023)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
por: Manzoni, Andrea
Publicado: (2026)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
por: Cao, Shiyi, et al.
Publicado: (2024)

Samoyeds: Accelerating MoE Models with Structured Sparsity Leveraging Sparse Tensor Cores
por: Wu, Chenpeng, et al.
Publicado: (2025)

Expert Divergence Learning for MoE-based Language Models
por: Li, Jiaang, et al.
Publicado: (2026)

Llama 3 Meets MoE: Efficient Upcycling
por: Vavre, Aditya, et al.
Publicado: (2024)

FluxMoE: Decoupling Expert Residency for High-Performance MoE Serving
por: Liu, Qingxiu, et al.
Publicado: (2026)

MoE-Prefill: Zero Redundancy Overheads in MoE Prefill Serving
por: Su, Zhaoyuan, et al.
Publicado: (2026)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
por: Ma, Songkai, et al.
Publicado: (2025)

MoE-GPS: Guidlines for Prediction Strategy for Dynamic Expert Duplication in MoE Load Balancing
por: Ma, Haiyue, et al.
Publicado: (2025)

Fast MoE Inference via Predictive Prefetching and Expert Replication
por: Jyothish, Ankit, et al.
Publicado: (2026)

Horseshoe Mixtures-of-Experts (HS-MoE)
por: Polson, Nick, et al.
Publicado: (2026)

PWC-MoE: Privacy-Aware Wireless Collaborative Mixture of Experts
por: Su, Yang, et al.
Publicado: (2025)

Geometric Asymmetry in MoE Specialization: Functional Decorrelation and Representational Overlap
por: Liu, Feilong
Publicado: (2026)

Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis
por: Pei, Zehua, et al.
Publicado: (2025)

$μ$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
por: Koike-Akino, Toshiaki, et al.
Publicado: (2025)

MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
por: Chen, Xiaodong, et al.
Publicado: (2025)

GRIN: GRadient-INformed MoE
por: Liu, Liyuan, et al.
Publicado: (2024)

Steering MoE LLMs via Expert (De)Activation
por: Fayyaz, Mohsen, et al.
Publicado: (2025)

FFT-MoE: Efficient Federated Fine-Tuning for Foundation Models via Large-scale Sparse MoE under Heterogeneous Edge
por: Hu, Gang, et al.
Publicado: (2025)