Cover Image

Model Merging Scaling Laws in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Yuanyi, Gu, Yanggan, Zhang, Yiming, Zhou, Qi, Yan, Zhaoyi, Xie, Congkai, Wang, Xinyao, Yuan, Jianbo, Yang, Hongxia
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MergePipe: A Budget-Aware Parameter Management System for Scalable LLM Merging
by: Wang, Yuanyi, et al.
Published: (2026)

Access Sets Matter: Budgeting Expert Reads for Scalable Weight-Space Model Merging
by: Wang, Yuanyi, et al.
Published: (2026)

InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Models
by: Gu, Yanggan, et al.
Published: (2025)

FeatCal: Feature Calibration for Post-Merging Models
by: Gu, Yanggan, et al.
Published: (2026)

InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion
by: Wang, Yuanyi, et al.
Published: (2025)

Infi-MMR: Curriculum-based Unlocking Multimodal Reasoning via Phased Reinforcement Learning in Multimodal Small Language Models
by: Liu, Zeyu, et al.
Published: (2025)

InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning
by: Xie, Congkai, et al.
Published: (2025)

InfiR2: A Comprehensive FP8 Training Recipe for Reasoning-Enhanced Language Models
by: Wang, Wenjun, et al.
Published: (2025)

InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Training
by: Wang, Pengkai, et al.
Published: (2025)

E-PMQ: Expert-Guided Post-Merge Quantization with Merged-Weight Anchoring
by: Wang, Wenjun, et al.
Published: (2026)

InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy Optimization
by: Liu, Yuhang, et al.
Published: (2025)

InfiAlign: A Scalable and Sample-Efficient Framework for Aligning LLMs to Enhance Reasoning Capabilities
by: Cai, Shuo, et al.
Published: (2025)

Reasoning Factual Knowledge in Structured Data with Large Language Models
by: Huang, Sirui, et al.
Published: (2024)

DavIR: Data Selection via Implicit Reward for Large Language Models
by: Zhou, Haotian, et al.
Published: (2023)

InfiMed-Foundation: Pioneering Advanced Multimodal Medical Models with Compute-Efficient Pre-Training and Multi-Stage Fine-Tuning
by: Zhu, Guanghao, et al.
Published: (2025)

Transport and Merge: Cross-Architecture Merging for Large Language Models
by: Cui, Chenhang, et al.
Published: (2026)

Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training
by: Wang, Yuanyi, et al.
Published: (2026)

Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning
by: Li, Zhen, et al.
Published: (2025)

InfiMed: Low-Resource Medical MLLMs with Advancing Understanding and Reasoning
by: Liu, Zeyu, et al.
Published: (2025)

Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models
by: Ye, Haoran, et al.
Published: (2024)

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models
by: Gu, Yanggan, et al.
Published: (2025)

Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
by: Wu, Yangzhen, et al.
Published: (2024)

The Law of Multi-Model Collaboration: Scaling Limits of Model Ensembling for Large Language Models
by: Lu, Dakuan, et al.
Published: (2025)

Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging
by: Cong, Tianshuo, et al.
Published: (2024)

Predictable Scale: Part II, Farseer: A Refined Scaling Law in Large Language Models
by: Li, Houyi, et al.
Published: (2025)

Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models
by: Ye, Haoran, et al.
Published: (2025)

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
by: Yang, Jinluan, et al.
Published: (2025)

Large Language Models for Predictive Analysis: How Far Are They?
by: Chen, Qin, et al.
Published: (2025)

Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
by: Wang, Yiqi, et al.
Published: (2024)

Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Models
by: Liu, Shuqi, et al.
Published: (2025)

Scaling Laws Across Model Architectures: A Comparative Analysis of Dense and MoE Models in Large Language Models
by: Wang, Siqi, et al.
Published: (2024)

AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Merging
by: Zhao, Yiran, et al.
Published: (2024)

Predictable Scale: Part I, Step Law -- Optimal Hyperparameter Scaling Law in Large Language Model Pretraining
by: Li, Houyi, et al.
Published: (2025)

LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Merging
by: Liu, Zehua, et al.
Published: (2025)

DPPA: Pruning Method for Large Language Model to Model Merging
by: Zhu, Yaochen, et al.
Published: (2024)

Refine Large Language Model Fine-tuning via Instruction Vector
by: Jiang, Gangwei, et al.
Published: (2024)

Chinese Labor Law Large Language Model Benchmark
by: Lan, Zixun, et al.
Published: (2026)

Quantization Meets Reasoning: Exploring and Mitigating Degradation of Low-Bit LLMs in Mathematical Reasoning
by: Li, Zhen, et al.
Published: (2025)

How to inject knowledge efficiently? Knowledge Infusion Scaling Law for Pre-training Large Language Models
by: Lv, Kangtao, et al.
Published: (2025)

InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners
by: Liu, Yuhang, et al.
Published: (2025)