Cover Image

Vanishing Feature: Diagnosing Model Merging and Beyond

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qu, Xingyu, Horvath, Samuel
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Superpose Task-specific Features for Model Merging
by: Qiu, Haiquan, et al.
Published: (2025)

FeatCal: Feature Calibration for Post-Merging Models
by: Gu, Yanggan, et al.
Published: (2026)

MIN-Merging: Merge the Important Neurons for Model Merging
by: Liang, Yunfei
Published: (2025)

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
by: Lu, Zhenyi, et al.
Published: (2024)

FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization
by: Chen, Hao Mark, et al.
Published: (2025)

PSO-Merging: Merging Models Based on Particle Swarm Optimization
by: Zhang, Kehao, et al.
Published: (2025)

Bayesian Model Merging
by: Li, Kaiyang, et al.
Published: (2026)

On Vanishing Variance in Transformer Length Generalization
by: Li, Ruining, et al.
Published: (2025)

Vanishing Contributions: A Unified Framework for Smooth and Iterative Model Compression
by: Nikiforos, Lorenzo, et al.
Published: (2025)

Merging Smarter, Generalizing Better: Enhancing Model Merging on OOD Data
by: Zhang, Bingjie, et al.
Published: (2025)

Decom-Renorm-Merge: Model Merging on the Right Space Improves Multitasking
by: Chaichana, Yuatyong, et al.
Published: (2025)

Model Merging: Foundations and Algorithms
by: Crisostomi, Donato
Published: (2026)

Model Merging in the Essential Subspace
by: Li, Longhua, et al.
Published: (2026)

Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness
by: Zhang, Qi, et al.
Published: (2024)

Can Muon Fine-tune Adam-Pretrained Models?
by: Qu, Xingyu, et al.
Published: (2026)

CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging
by: Sun, Wenju, et al.
Published: (2025)

Towards Minimizing Feature Drift in Model Merging: Layer-wise Task Vector Fusion for Adaptive Knowledge Integration
by: Sun, Wenju, et al.
Published: (2025)

Vanishing Gradients in Reinforcement Finetuning of Language Models
by: Razin, Noam, et al.
Published: (2023)

MergeMix: Optimizing Mid-Training Data Mixtures via Learnable Model Merging
by: Wang, Jiapeng, et al.
Published: (2026)

BD-Merging: Bias-Aware Dynamic Model Merging with Evidence-Guided Contrastive Learning
by: Xie, Yuhan, et al.
Published: (2026)

CALM: Consensus-Aware Localized Merging for Multi-Task Learning
by: Yan, Kunda, et al.
Published: (2025)

Training-free Heterogeneous Model Merging
by: Xu, Zhengqi, et al.
Published: (2024)

Revisiting Weight Averaging for Model Merging
by: Choi, Jiho, et al.
Published: (2024)

Multi-Level Collaboration in Model Merging
by: Li, Qi, et al.
Published: (2025)

Sparsity-Aware Evolution for Model Merging
by: Zhang, Huan, et al.
Published: (2026)

Fine, I'll Merge It Myself: A Multi-Fidelity Framework for Automated Model Merging
by: Su, Guinan, et al.
Published: (2025)

These Are Not All the Features You Are Looking For: A Fundamental Bottleneck in Supervised Pretraining
by: Yang, Xingyu Alice, et al.
Published: (2025)

Markov Chain Decoders Overcome the Heavy-Tail Limitations of Lipschitz Generative Models
by: Ziani, Abdelhakim, et al.
Published: (2026)

MergeIT: From Selection to Merging for Efficient Instruction Tuning
by: Cai, Hongyi, et al.
Published: (2025)

Redefining Contributions: Shapley-Driven Federated Learning
by: Tastan, Nurbek, et al.
Published: (2024)

Non-Uniform Parameter-Wise Model Merging
by: Camacho, Albert Manuel Orozco, et al.
Published: (2024)

Fair Streaming Feature Selection
by: Duan, Zhangling, et al.
Published: (2024)

Merge to Mix: Mixing Datasets via Model Merging
by: Tao, Zhixu Silvia, et al.
Published: (2025)

NegMerge: Sign-Consensual Weight Merging for Machine Unlearning
by: Kim, Hyo Seo, et al.
Published: (2024)

Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making
by: Hsu, Aliyah R., et al.
Published: (2023)

Toward a Holistic Approach to Continual Model Merging
by: Phan, Hoang, et al.
Published: (2025)

ToMA: Token Merge with Attention for Diffusion Models
by: Lu, Wenbo, et al.
Published: (2025)

Model Merging on Loss Landscape: A Geometry Perspective
by: Lu, Juanwu, et al.
Published: (2026)

Harnessing Optimization Dynamics for Curvature-Informed Model Merging
by: Mahdavinia, Pouria, et al.
Published: (2025)

Model Merging via Multi-Teacher Knowledge Distillation
by: Dalili, Seyed Arshan, et al.
Published: (2025)