Cover Image

Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Peng, Xujun, Kumar, Anoop, Wu, Jingyu, Glenn, Parker, Liu, Daben
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Improving Consistency in Retrieval-Augmented Systems with Group Similarity Rewards
by: Hamman, Faisal, et al.
Published: (2025)

Readability Reconsidered: A Cross-Dataset Analysis of Reference-Free Metrics
by: Belem, Catarina G, et al.
Published: (2025)

A Comparison of Independent and Joint Fine-tuning Strategies for Retrieval-Augmented Generation
by: Lawton, Neal Gregory, et al.
Published: (2025)

Play by the Type Rules: Inferring Constraints for LLM Functions in Declarative Programs
by: Glenn, Parker, et al.
Published: (2025)

LLM Optimization Unlocks Real-Time Pairwise Reranking
by: Wu, Jingyu, et al.
Published: (2025)

FB-RAG: Improving RAG with Forward and Backward Lookup
by: Chawla, Kushal, et al.
Published: (2025)

DF-RAG: Query-Aware Diversity for Retrieval-Augmented Generation
by: Khan, Saadat Hasan, et al.
Published: (2026)

Confidence-Based Response Abstinence: Improving LLM Trustworthiness via Activation-Based Uncertainty Estimation
by: Huang, Zhiqi, et al.
Published: (2025)

Alignment-Weighted DPO: A principled reasoning approach to improve safety alignment
by: Hu, Mengxuan, et al.
Published: (2026)

Refusal Tokens: A Simple Way to Calibrate Refusals in Large Language Models
by: Jain, Neel, et al.
Published: (2024)

Leveraging LLMs for Dialogue Quality Measurement
by: Jia, Jinghan, et al.
Published: (2024)

EMS: Adaptive Evict-then-Merge Strategy for Head-wise KV Cache Compression Based on Global-Local Importance
by: Li, Yingxin, et al.
Published: (2024)

Reasoning Resides in Layers: Restoring Temporal Reasoning in Video-Language Models with Layer-Selective Merging
by: Fu, Zihang, et al.
Published: (2026)

RAFFLES: Reasoning-based Attribution of Faults for LLM Systems
by: Zhu, Chenyang, et al.
Published: (2025)

Prompt Perturbation Consistency Learning for Robust Language Models
by: Qiang, Yao, et al.
Published: (2024)

Improve Decoding Factuality by Token-wise Cross Layer Entropy of Large Language Models
by: Wu, Jialiang, et al.
Published: (2025)

1bit-Merging: Dynamic Quantized Merging for Large Language Models
by: Liu, Shuqi, et al.
Published: (2025)

LEAD: Layer-wise Expert-aligned Decoding for Faithful Radiology Report Generation
by: Yang, Ruixiao, et al.
Published: (2026)

Layer-wise Regularized Dropout for Neural Language Models
by: Ni, Shiwen, et al.
Published: (2024)

Layer-wise Swapping for Generalizable Multilingual Safety
by: Shin, Hyunseo, et al.
Published: (2026)

Layer-wise Positional Bias in Short-Context Language Modeling
by: Rahimi, Maryam, et al.
Published: (2026)

Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
by: Tang, Yao, et al.
Published: (2026)

Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging
by: Liu, Deyuan, et al.
Published: (2024)

TruthTorchLM: A Comprehensive Library for Predicting Truthfulness in LLM Outputs
by: Yaldiz, Duygu Nur, et al.
Published: (2025)

Derivational Probing: Unveiling the Layer-wise Derivation of Syntactic Structures in Neural Language Models
by: Someya, Taiga, et al.
Published: (2025)

Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction
by: Liu, Zhexiong, et al.
Published: (2025)

Iterative Layer-wise Distillation for Efficient Compression of Large Language Models
by: Kovalev, Grigory, et al.
Published: (2025)

On the Effect of Uncertainty on Layer-wise Inference Dynamics
by: Kim, Sunwoo, et al.
Published: (2025)

ACE-Merging: Data-Free Model Merging with Adaptive Covariance Estimation
by: Xu, Bo, et al.
Published: (2026)

Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Models
by: Bachu, Saketh, et al.
Published: (2024)

Activation-Guided Consensus Merging for Large Language Models
by: Yao, Yuxuan, et al.
Published: (2025)

Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models
by: Lu, Jinliang, et al.
Published: (2024)

Uncertainty as Feature Gaps: Epistemic Uncertainty Quantification of LLMs in Contextual Question-Answering
by: Bakman, Yavuz, et al.
Published: (2025)

Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Models
by: Liu, Shuqi, et al.
Published: (2025)

Model Assembly Learning with Heterogeneous Layer Weight Merging
by: Zhang, Yi-Kai, et al.
Published: (2025)

GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment
by: Tang, Jiwei, et al.
Published: (2025)

Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning
by: Aakanksha, et al.
Published: (2024)

LaDiMo: Layer-wise Distillation Inspired MoEfier
by: Kim, Sungyoon, et al.
Published: (2024)

BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebra
by: Glenn, Parker, et al.
Published: (2024)

Uncertainty-Aware Step-wise Verification with Generative Reward Models
by: Ye, Zihuiwen, et al.
Published: (2025)