Imagen de Portada

Rethinking LLM Ensembling from the Perspective of Mixture Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Fu, Jiale, Jiang, Yuchu, Wu, Peijun, Liu, Chonghan, Zhou, Joey Tianyi, Yang, Xu
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Fast Large Language Model Collaborative Decoding via Speculation
por: Fu, Jiale, et al.
Publicado: (2025)

Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free
por: Li, Ziyue, et al.
Publicado: (2024)

d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
por: Jiang, Yuchu, et al.
Publicado: (2025)

Rethinking Data Mixing from the Perspective of Large Language Models
por: Xu, Yuanjian, et al.
Publicado: (2026)

Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?
por: Li, Wenzhe, et al.
Publicado: (2025)

Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory
por: Liu, Yexiang, et al.
Publicado: (2025)

DiscoUQ: Structured Disagreement Analysis for Uncertainty Quantification in LLM Agent Ensembles
por: Jiang, Bo
Publicado: (2026)

LLM-TOPLA: Efficient LLM Ensemble by Maximising Diversity
por: Tekin, Selim Furkan, et al.
Publicado: (2024)

Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theory
por: Li, Ming, et al.
Publicado: (2025)

Pre-trained Large Language Models Use Fourier Features to Compute Addition
por: Zhou, Tianyi, et al.
Publicado: (2024)

MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Models
por: Liu, Zehua, et al.
Publicado: (2025)

LLM Router: Rethinking Routing with Prefill Activations
por: Varshney, Tanay, et al.
Publicado: (2026)

Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Training
por: Shi, Hengyu, et al.
Publicado: (2026)

Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
por: Pan, Bowen, et al.
Publicado: (2024)

Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?
por: Wang, Shaobo, et al.
Publicado: (2025)

Rethinking Machine Unlearning for Large Language Models
por: Liu, Sijia, et al.
Publicado: (2024)

Rethinking Token Prediction: Tree-Structured Diffusion Language Model
por: Wu, Zihao, et al.
Publicado: (2026)

SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture
por: Han, Jiayi, et al.
Publicado: (2024)

Automated Alignment of Math Items to Content Standards in Large-Scale Assessments Using Language Models
por: Xu, Qingshu, et al.
Publicado: (2025)

Rethinking Misalignment in Vision-Language Model Adaptation from a Causal Perspective
por: Zhang, Yanan, et al.
Publicado: (2024)

When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification
por: Zhao, Jiale, et al.
Publicado: (2026)

Rethinking Token Reduction for State Space Models
por: Zhan, Zheng, et al.
Publicado: (2024)

PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective
por: Huang, Yangyi, et al.
Publicado: (2026)

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
por: Qiu, Wenjie, et al.
Publicado: (2025)

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
por: Zheng, Chen, et al.
Publicado: (2025)

S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning
por: Zeng, Hanqing, et al.
Publicado: (2025)

Flatter Tokens are More Valuable for Speculative Draft Model Training
por: Fan, Jiaming, et al.
Publicado: (2026)

MoPD: Mixture-of-Prompts Distillation for Vision-Language Models
por: Chen, Yang, et al.
Publicado: (2024)

A Closer Look into Mixture-of-Experts in Large Language Models
por: Lo, Ka Man, et al.
Publicado: (2024)

RevOrder: A Novel Method for Enhanced Arithmetic in Language Models
por: Shen, Si, et al.
Publicado: (2024)

CoMoE: Contrastive Representation for Mixture-of-Experts in Parameter-Efficient Fine-tuning
por: Feng, Jinyuan, et al.
Publicado: (2025)

OMoE: Diversifying Mixture of Low-Rank Adaptation by Orthogonal Finetuning
por: Feng, Jinyuan, et al.
Publicado: (2025)

Rethinking LLM Memorization through the Lens of Adversarial Compression
por: Schwarzschild, Avi, et al.
Publicado: (2024)

A Survey on Mixture of Experts in Large Language Models
por: Cai, Weilin, et al.
Publicado: (2024)

What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective
por: Li, Ming, et al.
Publicado: (2024)

GeoReasoner: Reasoning On Geospatially Grounded Context For Natural Language Understanding
por: Yan, Yibo, et al.
Publicado: (2024)

HMoE: Heterogeneous Mixture of Experts for Language Modeling
por: Wang, An, et al.
Publicado: (2024)

Critical Data Size of Language Models from a Grokking Perspective
por: Zhu, Xuekai, et al.
Publicado: (2024)

Kinetics: Rethinking Test-Time Scaling Laws
por: Sadhukhan, Ranajoy, et al.
Publicado: (2025)

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
por: Wang, Zihan, et al.
Publicado: (2025)