Cover Image

Channel Merging: Preserving Specialization for Merged Experts

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Mingyang, Liu, Jing, Ding, Ganggui, Yu, Xinyi, Ou, Linlin, Zhuang, Bohan
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models
by: Liu, Jing, et al.
Published: (2024)

Merge-of-Thought Distillation
by: Shen, Zhanming, et al.
Published: (2025)

Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging
by: Hui, Tingfeng, et al.
Published: (2024)

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
by: Lu, Zhenyi, et al.
Published: (2024)

REAM: Merging Improves Pruning of Experts in LLMs
by: Jha, Saurav, et al.
Published: (2026)

XFT: Unlocking the Power of Code Instruction Tuning by Simply Merging Upcycled Mixture-of-Experts
by: Ding, Yifeng, et al.
Published: (2024)

Merge to Mix: Mixing Datasets via Model Merging
by: Tao, Zhixu Silvia, et al.
Published: (2025)

Optimal Brain Iterative Merging: Mitigating Interference in LLM Merging
by: Wang, Zhixiang, et al.
Published: (2025)

Arcee's MergeKit: A Toolkit for Merging Large Language Models
by: Goddard, Charles, et al.
Published: (2024)

K-Merge: Online Continual Merging of Adapters for On-device Large Language Models
by: Shenaj, Donald, et al.
Published: (2025)

Model Merging for Knowledge Editing
by: Fu, Zichuan, et al.
Published: (2025)

Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation
by: Gauthier-Caron, Thomas, et al.
Published: (2024)

STAR: Spectral Truncation and Rescale for Model Merging
by: Lee, Yu-Ang, et al.
Published: (2025)

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
by: Yang, Jinluan, et al.
Published: (2025)

Parameter Competition Balancing for Model Merging
by: Du, Guodong, et al.
Published: (2024)

What Matters for Model Merging at Scale?
by: Yadav, Prateek, et al.
Published: (2024)

Dynamic Model Merging Made Slim
by: Du, Guodong, et al.
Published: (2026)

Model Assembly Learning with Heterogeneous Layer Weight Merging
by: Zhang, Yi-Kai, et al.
Published: (2025)

Fisher Mask Nodes for Language Model Merging
by: K, Thennal D, et al.
Published: (2024)

Model Merging by Uncertainty-Based Gradient Matching
by: Daheim, Nico, et al.
Published: (2023)

Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Merging
by: Zhang, Haobo, et al.
Published: (2025)

Merging Text Transformer Models from Different Initializations
by: Verma, Neha, et al.
Published: (2024)

Training-free LLM Merging for Multi-task Learning
by: Fu, Zichuan, et al.
Published: (2025)

Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
by: Li, Pingzhi, et al.
Published: (2023)

Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
by: Tang, Yao, et al.
Published: (2026)

Grow Up and Merge: Scaling Strategies for Efficient Language Adaptation
by: Glocker, Kevin, et al.
Published: (2025)

Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging
by: Ju, Yiming, et al.
Published: (2024)

SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data
by: Li, Jialu, et al.
Published: (2024)

Model Merging and Safety Alignment: One Bad Model Spoils the Bunch
by: Hammoud, Hasan Abed Al Kader, et al.
Published: (2024)

Data-driven Clustering and Merging of Adapters for On-device Large Language Models
by: Bohdal, Ondrej, et al.
Published: (2026)

Branch-Solve-Merge Improves Large Language Model Evaluation and Generation
by: Saha, Swarnadeep, et al.
Published: (2023)

Bone Soups: A Seek-and-Soup Model Merging Approach for Controllable Multi-Objective Generation
by: Xie, Guofu, et al.
Published: (2025)

SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
by: Zhuang, Haomin, et al.
Published: (2024)

FinTeamExperts: Role Specialized MOEs For Financial Analysis
by: Yu, Yue, et al.
Published: (2024)

Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs
by: Kadhe, Swanand Ravindra, et al.
Published: (2024)

MrT5: Dynamic Token Merging for Efficient Byte-level Language Models
by: Kallini, Julie, et al.
Published: (2024)

Linear Model Merging Unlocks Simple and Scalable Multimodal Data Mixture Optimization
by: Berasi, Davide, et al.
Published: (2026)

LoRA on the Go: Instance-level Dynamic LoRA Selection and Merging
by: Lee, Seungeon, et al.
Published: (2025)

Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering
by: Zhao, Ziyu, et al.
Published: (2024)

EvoMerge: Neuroevolution for Large Language Models
by: Jiang, Yushu
Published: (2024)