Cover Image

Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Ziyue, Zhou, Tianyi
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
by: Zheng, Chen, et al.
Published: (2025)

Geometry-Preserving Aggregation for Mixture-of-Experts Embedding Models
by: Kachuee, Sajjad, et al.
Published: (2026)

LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting
by: Zhang, Lingzheng, et al.
Published: (2024)

Rethinking LLM Ensembling from the Perspective of Mixture Models
by: Fu, Jiale, et al.
Published: (2026)

Routing-Free Mixture-of-Experts
by: Liu, Yilun, et al.
Published: (2026)

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
by: Wang, Zihan, et al.
Published: (2025)

It Takes Two: Your GRPO Is Secretly DPO
by: Wu, Yihong, et al.
Published: (2025)

Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data
by: Ou, Jingyang, et al.
Published: (2024)

Routing Manifold Alignment Improves Generalization of Mixture-of-Experts LLMs
by: Li, Zhongyang, et al.
Published: (2025)

Beyond instruction-conditioning, MoTE: Mixture of Task Experts for Multi-task Embedding Models
by: Romero, Miguel, et al.
Published: (2025)

Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
by: Wang, Lean, et al.
Published: (2024)

Maximum Score Routing For Mixture-of-Experts
by: Dong, Bowen, et al.
Published: (2025)

Mixture of Lookup Experts
by: Jie, Shibo, et al.
Published: (2025)

CoMoE: Contrastive Representation for Mixture-of-Experts in Parameter-Efficient Fine-tuning
by: Feng, Jinyuan, et al.
Published: (2025)

Your Transformer is Secretly Linear
by: Razzhigaev, Anton, et al.
Published: (2024)

R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts
by: Li, Zhongyang, et al.
Published: (2025)

LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation
by: Zhang, Xuan, et al.
Published: (2024)

Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models
by: Belenki, Lior, et al.
Published: (2025)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
by: Huang, Wei, et al.
Published: (2024)

MoIN: Mixture of Introvert Experts to Upcycle an LLM
by: Tejankar, Ajinkya, et al.
Published: (2024)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
by: Lu, Xudong, et al.
Published: (2024)

Contrastive Learning and Mixture of Experts Enables Precise Vector Embeddings
by: Hallee, Logan, et al.
Published: (2024)

HMoE: Heterogeneous Mixture of Experts for Language Modeling
by: Wang, An, et al.
Published: (2024)

Bayesian Mixture of Experts For Large Language Models
by: Dialameh, Maryam, et al.
Published: (2025)

How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradients
by: Li, Ming, et al.
Published: (2025)

Check Your LLM's Secret Dictionary! Five Lines of Code Reveal What Your LLM Learned (Including What It Shouldn't Have)
by: Miyashita, Hisashi
Published: (2026)

Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
by: He, Haoze, et al.
Published: (2026)

When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
by: Yoon, Youngsik, et al.
Published: (2026)

A Survey on Mixture of Experts in Large Language Models
by: Cai, Weilin, et al.
Published: (2024)

Scaling Laws for Upcycling Mixture-of-Experts Language Models
by: Liew, Seng Pei, et al.
Published: (2025)

Your Absorbing Discrete Diffusion Secretly Models the Bayesian Posterior
by: Doyle, Cooper
Published: (2025)

Slicing and Dicing: Configuring Optimal Mixtures of Experts
by: Li, Margaret, et al.
Published: (2026)

Binary-Integer-Programming Based Algorithm for Expert Load Balancing in Mixture-of-Experts Models
by: Sun, Yuan
Published: (2025)

Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline
by: Lee, Tony, et al.
Published: (2026)

FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models
by: Zhao, Zhongyu, et al.
Published: (2024)

Routing with Generated Data: Annotation-Free LLM Skill Estimation and Expert Selection
by: Niu, Tianyi, et al.
Published: (2026)

Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss
by: Lv, Ang, et al.
Published: (2025)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
by: Takashiro, Shota, et al.
Published: (2026)

A Closer Look into Mixture-of-Experts in Large Language Models
by: Lo, Ka Man, et al.
Published: (2024)

One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts
by: Wang, Ruochen, et al.
Published: (2024)