Cover Image

Orthogonal Model Merging

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Sihan, Shi, Kexuan, Liu, Weiyang
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Model Merging with Functional Dual Anchors
by: Shi, Kexuan, et al.
Published: (2025)

Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
by: Shi, Kexuan, et al.
Published: (2026)

POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformation
by: Qiu, Zeju, et al.
Published: (2026)

AMO: Adaptive Muon Orthogonalization
by: Zhuang, Xinlin, et al.
Published: (2026)

Joint User Priority and Power Scheduling for QoS-Aware WMMSE Precoding: A Constrained-Actor Attentive-Critic Approach
by: Wang, Kexuan, et al.
Published: (2025)

A Lightweight RL-Driven Deep Unfolding Network for Robust WMMSE Precoding in Massive MU-MIMO-OFDM Systems
by: Wang, Kexuan, et al.
Published: (2025)

Orthogonal Finetuning Made Scalable
by: Qiu, Zeju, et al.
Published: (2025)

MOMA: Masked Orthogonal Matrix Alignment for Zero-Additional-Parameter Model Merging
by: Kong, Fanshuang, et al.
Published: (2024)

A Single-Loop Deep Actor-Critic Algorithm for Constrained Reinforcement Learning with Provable Convergence
by: Wang, Kexuan, et al.
Published: (2023)

Context-aware Constrained Reinforcement Learning Based Energy-Efficient Power Scheduling for Non-stationary XR Data Traffic
by: Wang, Kexuan, et al.
Published: (2025)

Reparameterized LLM Training via Orthogonal Equivalence Transformation
by: Qiu, Zeju, et al.
Published: (2025)

Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Merging
by: Zhang, Haobo, et al.
Published: (2025)

Caformer: Rethinking Time Series Analysis from Causal Perspective
by: Zhang, Kexuan, et al.
Published: (2024)

Causal Learning for Heterogeneous Subgroups Based on Nonlinear Causal Kernel Clustering
by: Liu, Lu, et al.
Published: (2025)

Modular Delta Merging with Orthogonal Constraints: A Scalable Framework for Continual and Reversible Model Composition
by: Khan, Haris, et al.
Published: (2025)

Behavior Knowledge Merge in Reinforced Agentic Models
by: Yuan, Xiangchi, et al.
Published: (2026)

Merge and Guide: Unifying Model Merging and Guided Decoding for Controllable Multi-Objective Generation
by: Xie, Guofu, et al.
Published: (2025)

MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
by: Miao, Ruijie, et al.
Published: (2025)

PSO-Merging: Merging Models Based on Particle Swarm Optimization
by: Zhang, Kehao, et al.
Published: (2025)

Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging
by: Tang, Anke, et al.
Published: (2025)

Decouple and Orthogonalize: A Data-Free Framework for LoRA Merging
by: Zheng, Shenghe, et al.
Published: (2025)

Why Do More Experts Fail? A Theoretical Analysis of Model Merging
by: Wang, Zijing, et al.
Published: (2025)

AURORA: Contextual Orthogonalization for Geometric Representation Learning in Healthcare Foundation Models
by: Zhang, Yuanyun, et al.
Published: (2026)

BadMerging: Backdoor Attacks Against Model Merging
by: Zhang, Jinghuai, et al.
Published: (2024)

Controlling Text-to-Image Diffusion by Orthogonal Finetuning
by: Qiu, Zeju, et al.
Published: (2023)

Merging Smarter, Generalizing Better: Enhancing Model Merging on OOD Data
by: Zhang, Bingjie, et al.
Published: (2025)

FedMerge: Federated Personalization via Model Merging
by: Chen, Shutong, et al.
Published: (2025)

MIN-Merging: Merge the Important Neurons for Model Merging
by: Liang, Yunfei
Published: (2025)

Rethinking Inter-LoRA Orthogonality in Adapter Merging: Insights from Orthogonal Monte Carlo Dropout
by: Zhang, Andi, et al.
Published: (2025)

AdaMerging: Adaptive Model Merging for Multi-Task Learning
by: Yang, Enneng, et al.
Published: (2023)

Rethinking Layer-wise Model Merging through Chain of Merges
by: Buzzega, Pietro, et al.
Published: (2025)

From Task-Specific Models to Unified Systems: A Review of Model Merging Approaches
by: Ruan, Wei, et al.
Published: (2025)

Breaking the Reasoning Horizon in Entity Alignment Foundation Models
by: Cui, Yuanning, et al.
Published: (2026)

Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generation
by: Yang, Xinyu, et al.
Published: (2025)

Pareto Merging: Multi-Objective Optimization for Preference-Aware Model Merging
by: Chen, Weiyu, et al.
Published: (2024)

OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain
by: Yue, Wenzhen, et al.
Published: (2025)

MergeBench: A Benchmark for Merging Domain-Specialized LLMs
by: He, Yifei, et al.
Published: (2025)

Inductive Gradient Adjustment For Spectral Bias In Implicit Neural Representations
by: Shi, Kexuan, et al.
Published: (2024)

DRAN: A Distribution and Relation Adaptive Network for Spatio-temporal Forecasting
by: Zou, Xiaobei, et al.
Published: (2025)

Extra-Merge: Tracing the Rank-1 Subspace of Model Merging in Language Model Pre-Training
by: Zhou, Wenjie, et al.
Published: (2026)