Imagen de Portada

Geometry-Preserving Aggregation for Mixture-of-Experts Embedding Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Kachuee, Sajjad, Sharifkhani, Mohammad
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Efficient Large Language Models with Zero-Shot Adjustable Acceleration
por: Kachuee, Sajjad, et al.
Publicado: (2025)

Latency Adjustable Transformer Encoder for Language Understanding
por: Kachuee, Sajjad, et al.
Publicado: (2022)

Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
por: He, Haoze, et al.
Publicado: (2026)

Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free
por: Li, Ziyue, et al.
Publicado: (2024)

Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
por: Ahrac, Sagi, et al.
Publicado: (2026)

Beyond instruction-conditioning, MoTE: Mixture of Task Experts for Multi-task Embedding Models
por: Romero, Miguel, et al.
Publicado: (2025)

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
por: Wang, Zihan, et al.
Publicado: (2025)

Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models
por: Belenki, Lior, et al.
Publicado: (2025)

Mixture of Lookup Experts
por: Jie, Shibo, et al.
Publicado: (2025)

Upcycling Large Language Models into Mixture of Experts
por: He, Ethan, et al.
Publicado: (2024)

Contrastive Learning and Mixture of Experts Enables Precise Vector Embeddings
por: Hallee, Logan, et al.
Publicado: (2024)

Bayesian Mixture of Experts For Large Language Models
por: Dialameh, Maryam, et al.
Publicado: (2025)

When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
por: Yoon, Youngsik, et al.
Publicado: (2026)

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
por: Zheng, Chen, et al.
Publicado: (2025)

Not Eliminate but Aggregate: Post-Hoc Control over Mixture-of-Experts to Address Shortcut Shifts in Natural Language Understanding
por: Honda, Ukyo, et al.
Publicado: (2024)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
por: Huang, Wei, et al.
Publicado: (2024)

A Survey on Mixture of Experts in Large Language Models
por: Cai, Weilin, et al.
Publicado: (2024)

Scaling Laws for Upcycling Mixture-of-Experts Language Models
por: Liew, Seng Pei, et al.
Publicado: (2025)

HMoE: Heterogeneous Mixture of Experts for Language Modeling
por: Wang, An, et al.
Publicado: (2024)

Binary-Integer-Programming Based Algorithm for Expert Load Balancing in Mixture-of-Experts Models
por: Sun, Yuan
Publicado: (2025)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
por: Takashiro, Shota, et al.
Publicado: (2026)

Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss
por: Lv, Ang, et al.
Publicado: (2025)

A Closer Look into Mixture-of-Experts in Large Language Models
por: Lo, Ka Man, et al.
Publicado: (2024)

Mixture of Experts Made Intrinsically Interpretable
por: Yang, Xingyi, et al.
Publicado: (2025)

Maximum Score Routing For Mixture-of-Experts
por: Dong, Bowen, et al.
Publicado: (2025)

Mixture-of-Experts as Soft Clustering: A Dual Jacobian-PCA Spectral Geometry Perspective
por: Liu, Feilong
Publicado: (2026)

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
por: Zhang, Zeliang, et al.
Publicado: (2024)

Skin-in-the-Game: Decision Making via Multi-Stakeholder Alignment in LLMs
por: Sel, Bilgehan, et al.
Publicado: (2024)

Mixture of LoRA Experts
por: Wu, Xun, et al.
Publicado: (2024)

MoSE: Mixture of Slimmable Experts for Efficient and Adaptive Language Models
por: Tastan, Nurbek, et al.
Publicado: (2026)

Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts
por: Wang, Haoxiang, et al.
Publicado: (2024)

MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Models
por: Liu, Zehua, et al.
Publicado: (2025)

Beyond Benchmarks: Understanding Mixture-of-Experts Models through Internal Mechanisms
por: Ying, Jiahao, et al.
Publicado: (2025)

Slicing and Dicing: Configuring Optimal Mixtures of Experts
por: Li, Margaret, et al.
Publicado: (2026)

MoFE: Mixture of Frozen Experts Architecture
por: Seo, Jean, et al.
Publicado: (2025)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
por: Muzio, Alexandre, et al.
Publicado: (2024)

Shared Global and Local Geometry of Language Model Embeddings
por: Lee, Andrew, et al.
Publicado: (2025)

Mixture of Heterogeneous Grouped Experts for Language Modeling
por: Ma, Zhicheng, et al.
Publicado: (2026)

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
por: Liu, Yang, et al.
Publicado: (2026)

Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations
por: Dong, Zican, et al.
Publicado: (2025)