Cover Image

MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Yuhang, Karamanolakis, Giannis, Soto, Victor, Rumshisky, Anna, Kulkarni, Mayank, Huang, Furong, Ai, Wei, Lu, Jianhua
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
by: Miao, Ruijie, et al.
Published: (2025)

Retraining-Free Merging of Sparse MoE via Hierarchical Clustering
by: Chen, I-Chun, et al.
Published: (2024)

Interactive Machine Teaching by Labeling Rules and Instances
by: Karamanolakis, Giannis, et al.
Published: (2024)

Faster, Smaller, and Smarter: Task-Aware Expert Merging for Online MoE Inference
by: Han, Ziyi, et al.
Published: (2025)

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
by: Li, Lujun, et al.
Published: (2025)

UMM-RM: An Upcycle-and-Merge MoE Reward Model for Mitigating Reward Hacking
by: Fu, Lingling, et al.
Published: (2025)

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning
by: Sinha, Sanchit, et al.
Published: (2024)

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
by: Lu, Zhenyi, et al.
Published: (2024)

GFairHint: Improving Individual Fairness for Graph Neural Networks via Fairness Hint
by: Xu, Paiheng, et al.
Published: (2023)

DC-Merge: Improving Model Merging with Directional Consistency
by: Zhang, Han-Chen, et al.
Published: (2026)

Training-free Heterogeneous Model Merging
by: Xu, Zhengqi, et al.
Published: (2024)

Enhancing Multi-modal Models with Heterogeneous MoE Adapters for Fine-tuning
by: Zhou, Sashuai, et al.
Published: (2025)

MIN-Merging: Merge the Important Neurons for Model Merging
by: Liang, Yunfei
Published: (2025)

PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging
by: Shao, Zibo, et al.
Published: (2026)

Sparse Crosscoders for diffing MoEs and Dense models
by: Chaudhari, Marmik, et al.
Published: (2026)

ASD-RSD; To Merge or Not to Merge
by: Sinclair, Dorothy
Published: (1971)

FedMerge: Federated Personalization via Model Merging
by: Chen, Shutong, et al.
Published: (2025)

MergeTok: Unified Continuous and Discrete Visual Tokenization via Token Merging
by: Zhang, Luyuan, et al.
Published: (2026)

Merge to Mix: Mixing Datasets via Model Merging
by: Tao, Zhixu Silvia, et al.
Published: (2025)

Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
by: Li, Bo, et al.
Published: (2026)

ATM: Improving Model Merging by Alternating Tuning and Merging
by: Zhou, Luca, et al.
Published: (2024)

SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations
by: Guo, Wentao, et al.
Published: (2025)

Explore Spurious Correlations at the Concept Level in Language Models for Text Classification
by: Zhou, Yuhang, et al.
Published: (2023)

Merge-of-Thought Distillation
by: Shen, Zhanming, et al.
Published: (2025)

To Merge and Not to Merge: Israel's Union List of Monographs in the Context of Merging Algorithms.
by: Lazinger, Susan S.
Published: (1994)

OptMerge: Unifying Multimodal LLM Capabilities and Modalities via Model Merging
by: Wei, Yongxian, et al.
Published: (2025)

EMR-Merging: Tuning-Free High-Performance Model Merging
by: Huang, Chenyu, et al.
Published: (2024)

Polysemantic Experts, Monosemantic Paths: Routing as Control in MoEs
by: Ye, Charles, et al.
Published: (2026)

MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token Merging
by: Li, Siyuan, et al.
Published: (2025)

Extra-Merge: Tracing the Rank-1 Subspace of Model Merging in Language Model Pre-Training
by: Zhou, Wenjie, et al.
Published: (2026)

Merge Hijacking: Backdoor Attacks to Model Merging of Large Language Models
by: Yuan, Zenghui, et al.
Published: (2025)

HM3: Heterogeneous Multi-Class Model Merging
by: Hackmann, Stefan
Published: (2024)

Model Assembly Learning with Heterogeneous Layer Weight Merging
by: Zhang, Yi-Kai, et al.
Published: (2025)

SE-Merging: A Self-Enhanced Approach for Dynamic Model Merging
by: Chen, Zijun, et al.
Published: (2025)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
by: Wu, Haoyuan, et al.
Published: (2025)

Extrapolation Merging: Keep Improving With Extrapolation and Merging
by: Lin, Yiguan, et al.
Published: (2025)

Channel Merging: Preserving Specialization for Merged Experts
by: Zhang, Mingyang, et al.
Published: (2024)

DeepMerge: Deep-Learning-Based Region-Merging for Image Segmentation
by: Lv, Xianwei, et al.
Published: (2023)

Merged Reference Desks in a Merged Library Environment: The Impact of Merging Government Publications
by: Kendall, Susan L., et al.
Published: (2006)

CSRec: Rethinking Sequential Recommendation from A Causal Perspective
by: Liu, Xiaoyu, et al.
Published: (2024)