Cover Image

Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wei, Yongxian, Tang, Anke, Shen, Li, Hu, Zixuan, Yuan, Chun, Cao, Xiaochun
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models
by: Wei, Yongxian, et al.
Published: (2024)

Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectors
by: Cheng, Runxi, et al.
Published: (2025)

Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging
by: Shen, Li, et al.
Published: (2024)

Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs
by: Hu, Zixuan, et al.
Published: (2024)

Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler
by: Hu, Zixuan, et al.
Published: (2025)

Task-Distributionally Robust Data-Free Meta-Learning
by: Hu, Zixuan, et al.
Published: (2023)

Merging Multi-Task Models via Weight-Ensembling Mixture of Experts
by: Tang, Anke, et al.
Published: (2024)

Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspace
by: Yang, Jinluan, et al.
Published: (2024)

Sparse Model Inversion: Efficient Inversion of Vision Transformers for Data-Free Applications
by: Hu, Zixuan, et al.
Published: (2025)

AdaMerging: Adaptive Model Merging for Multi-Task Learning
by: Yang, Enneng, et al.
Published: (2023)

FREE: Faster and Better Data-Free Meta-Learning
by: Wei, Yongxian, et al.
Published: (2024)

Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging
by: Tang, Anke, et al.
Published: (2025)

Multi-Task Model Merging via Adaptive Weight Disentanglement
by: Xiong, Feng, et al.
Published: (2024)

Attacking Large Language Models with Projected Gradient Descent
by: Geisler, Simon, et al.
Published: (2024)

A Unified Generalization Framework for Model Merging: Trade-offs, Non-Linearity, and Scaling Laws
by: Li, Qinglun, et al.
Published: (2026)

Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences
by: Cao, Jiarui, et al.
Published: (2026)

Transformers Learn to Implement Multi-step Gradient Descent with Chain of Thought
by: Huang, Jianhao, et al.
Published: (2025)

Representation Surgery for Multi-Task Model Merging
by: Yang, Enneng, et al.
Published: (2024)

One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planning
by: Pu, Yuan, et al.
Published: (2025)

Transformers Trained via Gradient Descent Can Provably Learn a Class of Teacher Models
by: Zhang, Chenyang, et al.
Published: (2026)

Grams: Gradient Descent with Adaptive Momentum Scaling
by: Cao, Yang, et al.
Published: (2024)

Stochastic Adaptive Gradient Descent Without Descent
by: Aujol, Jean-François, et al.
Published: (2025)

Transformers Efficiently Perform In-Context Logistic Regression via Normalized Gradient Descent
by: Zhang, Chenyang, et al.
Published: (2026)

Adaptive Conditional Gradient Descent
by: Khademi, Abbas, et al.
Published: (2025)

Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Masks
by: Davari, MohammadReza, et al.
Published: (2023)

OptMerge: Unifying Multimodal LLM Capabilities and Modalities via Model Merging
by: Wei, Yongxian, et al.
Published: (2025)

Enhancing Adversarial Text Attacks on BERT Models with Projected Gradient Descent
by: Waghela, Hetvi, et al.
Published: (2024)

Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities
by: Yang, Enneng, et al.
Published: (2024)

Stochastic Gradient Descent with Adaptive Data
by: Che, Ethan, et al.
Published: (2024)

Stochastic Normalized Gradient Descent with Momentum for Large-Batch Training
by: Zhao, Shen-Yi, et al.
Published: (2020)

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
by: Yang, Jinluan, et al.
Published: (2025)

Learning Tree-Based Models with Gradient Descent
by: Marton, Sascha
Published: (2026)

Bolstering Stochastic Gradient Descent with Model Building
by: Birbil, S. Ilker, et al.
Published: (2021)

Adaptive Kernel Selection for Stein Variational Gradient Descent
by: Melcher, Moritz, et al.
Published: (2025)

Elastic Multi-Gradient Descent for Parallel Continual Learning
by: Lyu, Fan, et al.
Published: (2024)

Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent
by: Deng, Xiaoge, et al.
Published: (2023)

Integrating Intermediate Layer Optimization and Projected Gradient Descent for Solving Inverse Problems with Diffusion Models
by: Zheng, Yang, et al.
Published: (2025)

Euclidean Distance Matrix Completion via Asymmetric Projected Gradient Descent
by: Li, Yicheng, et al.
Published: (2025)

The Implicit Bias of Steepest Descent with Mini-batch Stochastic Gradient
by: Li, Jichu, et al.
Published: (2026)

Parameter Efficient Multi-task Model Fusion with Partial Linearization
by: Tang, Anke, et al.
Published: (2023)