Copertina

Trade-offs in Ensembling, Merging and Routing Among Parameter-Efficient Experts

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Lotfi, Sanae, Caccia, Lucas, Sordoni, Alessandro, Ash, Jordan T., Dudik, Miroslav
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
di: Arnob, Samin Yeasar, et al.
Pubblicazione: (2025)

A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning
di: Yadav, Prateek, et al.
Pubblicazione: (2024)

Training Plug-n-Play Knowledge Modules with Deep Context Distillation
di: Caccia, Lucas, et al.
Pubblicazione: (2025)

Learning to Solve Complex Problems via Dataset Decomposition
di: Zhao, Wanru, et al.
Pubblicazione: (2026)

Generative Modeling of Individual Behavior at Scale
di: Omi, Nabil, et al.
Pubblicazione: (2025)

Learning to Extract Context for Context-Aware LLM Inference
di: Kim, Minseon, et al.
Pubblicazione: (2025)

Guiding Language Model Reasoning with Planning Tokens
di: Wang, Xinyi, et al.
Pubblicazione: (2023)

Soft Merging of Experts with Adaptive Routing
di: Muqeeth, Mohammed, et al.
Pubblicazione: (2023)

Astral Space: Convex Analysis at Infinity
di: Dudík, Miroslav, et al.
Pubblicazione: (2022)

On the Hardness of Bandit Learning
di: Brukhim, Nataly, et al.
Pubblicazione: (2025)

Breaking the Likelihood-Quality Trade-off in Diffusion Models by Merging Pretrained Experts
di: Esfandiari, Yasin, et al.
Pubblicazione: (2025)

Towards Modular LLMs by Building and Reusing a Library of LoRAs
di: Ostapenko, Oleksiy, et al.
Pubblicazione: (2024)

Quantized Reasoning Models Think They Need to Think Longer, but They Do Not
di: Lotfi, Sanae, et al.
Pubblicazione: (2026)

A structured regression approach for evaluating model performance across intersectional subgroups
di: Herlihy, Christine, et al.
Pubblicazione: (2024)

SureMap: Simultaneous Mean Estimation for Single-Task and Multi-Task Disaggregated Evaluation
di: Khodak, Mikhail, et al.
Pubblicazione: (2024)

Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging
di: Shen, Li, et al.
Pubblicazione: (2024)

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
di: Liu, Yilun, et al.
Pubblicazione: (2024)

HydraOpt: Navigating the Efficiency-Performance Trade-off of Adapter Merging
di: Ceritli, Taha, et al.
Pubblicazione: (2025)

Efficient Adversarial Training in LLMs with Continuous Attacks
di: Xhonneux, Sophie, et al.
Pubblicazione: (2024)

Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful
di: Marek, Martin, et al.
Pubblicazione: (2025)

Learning to Route Among Specialized Experts for Zero-Shot Generalization
di: Muqeeth, Mohammed, et al.
Pubblicazione: (2024)

MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
di: Miao, Ruijie, et al.
Pubblicazione: (2025)

Merging Multi-Task Models via Weight-Ensembling Mixture of Experts
di: Tang, Anke, et al.
Pubblicazione: (2024)

A Study of Plasticity Loss in On-Policy Deep Reinforcement Learning
di: Juliani, Arthur, et al.
Pubblicazione: (2024)

Expert Merging: Model Merging with Unsupervised Expert Alignment and Importance-Guided Layer Chunking
di: Zhang, Dengming, et al.
Pubblicazione: (2025)

A Unified Generalization Framework for Model Merging: Trade-offs, Non-Linearity, and Scaling Laws
di: Li, Qinglun, et al.
Pubblicazione: (2026)

Learning More Generalized Experts by Merging Experts in Mixture-of-Experts
di: Park, Sejik
Pubblicazione: (2024)

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
di: Li, Lujun, et al.
Pubblicazione: (2025)

IPR: Intelligent Prompt Routing with User-Controlled Quality-Cost Trade-offs
di: Feng, Aosong, et al.
Pubblicazione: (2025)

Parameter-Efficient Checkpoint Merging via Metrics-Weighted Averaging
di: Yu, Shi Jie, et al.
Pubblicazione: (2025)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
di: Liu, Yilun, et al.
Pubblicazione: (2025)

Expert Merging in Sparse Mixture of Experts with Nash Bargaining
di: Nguyen, Dung V., et al.
Pubblicazione: (2025)

Customizing the Inductive Biases of Softmax Attention using Structured Matrices
di: Kuang, Yilun, et al.
Pubblicazione: (2025)

Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Models
di: Lotfi, Sanae, et al.
Pubblicazione: (2024)

Ensembles at Any Cost? Accuracy-Energy Trade-offs in Recommender Systems
di: Nitschke, Jannik, et al.
Pubblicazione: (2026)

Not All LLM Reasoners Are Created Equal
di: Hosseini, Arian, et al.
Pubblicazione: (2024)

Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs
di: Wei, Wang, et al.
Pubblicazione: (2025)

Multilingual Routing in Mixture-of-Experts
di: Bandarkar, Lucas, et al.
Pubblicazione: (2025)

LLM-Based Routing in Mixture of Experts: A Novel Framework for Trading
di: Liu, Kuan-Ming, et al.
Pubblicazione: (2025)

CAMEx: Curvature-aware Merging of Experts
di: Nguyen, Dung V., et al.
Pubblicazione: (2025)