Cover Image

Merge to Mix: Mixing Datasets via Model Merging

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tao, Zhixu Silvia, Vinken, Kasper, Yeh, Hao-Wei, Cooper, Avi, Boix, Xavier
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
by: Yang, Jinluan, et al.
Published: (2025)

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
by: Lu, Zhenyi, et al.
Published: (2024)

STAR: Spectral Truncation and Rescale for Model Merging
by: Lee, Yu-Ang, et al.
Published: (2025)

Rethinking VLMs and LLMs for Image Classification
by: Cooper, Avi, et al.
Published: (2024)

Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
by: Tang, Yao, et al.
Published: (2026)

Arcee's MergeKit: A Toolkit for Merging Large Language Models
by: Goddard, Charles, et al.
Published: (2024)

Channel Merging: Preserving Specialization for Merged Experts
by: Zhang, Mingyang, et al.
Published: (2024)

OptiMer: Optimal Distribution Vector Merging Is Better than Data Mixing for Continual Pre-Training
by: Song, Haiyue, et al.
Published: (2026)

Merge-of-Thought Distillation
by: Shen, Zhanming, et al.
Published: (2025)

K-Merge: Online Continual Merging of Adapters for On-device Large Language Models
by: Shenaj, Donald, et al.
Published: (2025)

Model Merging for Knowledge Editing
by: Fu, Zichuan, et al.
Published: (2025)

Optimal Brain Iterative Merging: Mitigating Interference in LLM Merging
by: Wang, Zhixiang, et al.
Published: (2025)

Dynamic Model Merging Made Slim
by: Du, Guodong, et al.
Published: (2026)

What Matters for Model Merging at Scale?
by: Yadav, Prateek, et al.
Published: (2024)

MergeMix: Optimizing Mid-Training Data Mixtures via Learnable Model Merging
by: Wang, Jiapeng, et al.
Published: (2026)

Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation
by: Gauthier-Caron, Thomas, et al.
Published: (2024)

Fisher Mask Nodes for Language Model Merging
by: K, Thennal D, et al.
Published: (2024)

Model Merging by Uncertainty-Based Gradient Matching
by: Daheim, Nico, et al.
Published: (2023)

Model Assembly Learning with Heterogeneous Layer Weight Merging
by: Zhang, Yi-Kai, et al.
Published: (2025)

Merging Text Transformer Models from Different Initializations
by: Verma, Neha, et al.
Published: (2024)

Model Merging and Safety Alignment: One Bad Model Spoils the Bunch
by: Hammoud, Hasan Abed Al Kader, et al.
Published: (2024)

Navigating the Alignment-Calibration Trade-off: A Pareto-Superior Frontier via Model Merging
by: Hu, Tiancheng, et al.
Published: (2025)

Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Merging
by: Zhang, Haobo, et al.
Published: (2025)

Data-driven Clustering and Merging of Adapters for On-device Large Language Models
by: Bohdal, Ondrej, et al.
Published: (2026)

Branch-Solve-Merge Improves Large Language Model Evaluation and Generation
by: Saha, Swarnadeep, et al.
Published: (2023)

Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging
by: Hui, Tingfeng, et al.
Published: (2024)

Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging
by: Ju, Yiming, et al.
Published: (2024)

Training-free LLM Merging for Multi-task Learning
by: Fu, Zichuan, et al.
Published: (2025)

EvoMerge: Neuroevolution for Large Language Models
by: Jiang, Yushu
Published: (2024)

Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs
by: Kadhe, Swanand Ravindra, et al.
Published: (2024)

MrT5: Dynamic Token Merging for Efficient Byte-level Language Models
by: Kallini, Julie, et al.
Published: (2024)

Linear Model Merging Unlocks Simple and Scalable Multimodal Data Mixture Optimization
by: Berasi, Davide, et al.
Published: (2026)

Parameter Competition Balancing for Model Merging
by: Du, Guodong, et al.
Published: (2024)

MAGIC: Achieving Superior Model Merging via Magnitude Calibration
by: Li, Yayuan, et al.
Published: (2025)

Multi-task Code LLMs: Data Mix or Model Merge?
by: Zhu, Mingzhi, et al.
Published: (2026)

Grow Up and Merge: Scaling Strategies for Efficient Language Adaptation
by: Glocker, Kevin, et al.
Published: (2025)

REAM: Merging Improves Pruning of Experts in LLMs
by: Jha, Saurav, et al.
Published: (2026)

Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities
by: Yang, Enneng, et al.
Published: (2024)

Bone Soups: A Seek-and-Soup Model Merging Approach for Controllable Multi-Objective Generation
by: Xie, Guofu, et al.
Published: (2025)

BiMix: A Bivariate Data Mixing Law for Language Model Pretraining
by: Ge, Ce, et al.
Published: (2024)