Imagen de Portada

Functionality-Oriented LLM Merging on the Fisher--Rao Manifold

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wang, Jiayu, Ye, Zuojun, Yin, Wenpeng
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Fisher Mask Nodes for Language Model Merging
por: K, Thennal D, et al.
Publicado: (2024)

Training-free LLM Merging for Multi-task Learning
por: Fu, Zichuan, et al.
Publicado: (2025)

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens
por: Kwek, Eugene, et al.
Publicado: (2025)

Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Merging
por: Zhang, Haobo, et al.
Publicado: (2025)

Optimal Brain Iterative Merging: Mitigating Interference in LLM Merging
por: Wang, Zhixiang, et al.
Publicado: (2025)

Understanding Dynamic Compute Allocation in Recurrent Transformers
por: Moosa, Ibraheem Muhammad, et al.
Publicado: (2026)

FGGM: Fisher-Guided Gradient Masking for Continual Learning
por: Tan, Chao-Hong, et al.
Publicado: (2026)

Boosting Adversarial Training via Fisher-Rao Norm-based Regularization
por: Yin, Xiangyu, et al.
Publicado: (2024)

Merging Beyond: Streaming LLM Updates via Activation-Guided Rotations
por: Yao, Yuxuan, et al.
Publicado: (2026)

Sampling in Unit Time with Kernel Fisher-Rao Flow
por: Maurais, Aimee, et al.
Publicado: (2024)

DOTResize: Reducing LLM Width via Discrete Optimal Transport-based Neuron Merging
por: Verma, Neha, et al.
Publicado: (2025)

Merge to Learn: Efficiently Adding Skills to Language Models with Model Merging
por: Morrison, Jacob, et al.
Publicado: (2024)

FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers
por: Wang, Sihan, et al.
Publicado: (2026)

Less Finetuning, Better Retrieval: Rethinking LLM Adaptation for Biomedical Retrievers via Synthetic Data and Model Merging
por: Khattab, Sameh, et al.
Publicado: (2026)

LLM Enhancer: Merged Approach using Vector Embedding for Reducing Large Language Model Hallucinations with External Knowledge
por: Rayhan, Naheed, et al.
Publicado: (2025)

Model Assembly Learning with Heterogeneous Layer Weight Merging
por: Zhang, Yi-Kai, et al.
Publicado: (2025)

Direct-Inverse Prompting: Analyzing LLMs' Discriminative Capacity in Self-Improving Generation
por: Ahn, Jihyun Janice, et al.
Publicado: (2024)

FLoE: Fisher-Based Layer Selection for Efficient Sparse Adaptation of Low-Rank Experts
por: Wang, Xinyi, et al.
Publicado: (2025)

Channel Merging: Preserving Specialization for Merged Experts
por: Zhang, Mingyang, et al.
Publicado: (2024)

NAN: A Training-Free Solution to Coefficient Estimation in Model Merging
por: Si, Chongjie, et al.
Publicado: (2025)

Incentivizing Agentic Reasoning in LLM Judges via Tool-Integrated Reinforcement Learning
por: Xu, Ran, et al.
Publicado: (2025)

Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging
por: Ju, Yiming, et al.
Publicado: (2024)

Merging Feed-Forward Sublayers for Compressed Transformers
por: Verma, Neha, et al.
Publicado: (2025)

Merging by Matching Models in Task Parameter Subspaces
por: Tam, Derek, et al.
Publicado: (2023)

Model Merging in Pre-training of Large Language Models
por: Li, Yunshui, et al.
Publicado: (2025)

Merge to Mix: Mixing Datasets via Model Merging
por: Tao, Zhixu Silvia, et al.
Publicado: (2025)

Model Merging for Knowledge Editing
por: Fu, Zichuan, et al.
Publicado: (2025)

MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
por: Qin, Jiayu, et al.
Publicado: (2025)

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
por: Lu, Zhenyi, et al.
Publicado: (2024)

Contrastive Instruction Tuning
por: Yan, Tianyi Lorena, et al.
Publicado: (2024)

Dataless Knowledge Fusion by Merging Weights of Language Models
por: Jin, Xisen, et al.
Publicado: (2022)

Emotion Collider: Dual Hyperbolic Mirror Manifolds for Sentiment Recovery via Anti Emotion Reflection
por: Fu, Rong, et al.
Publicado: (2026)

Arcee's MergeKit: A Toolkit for Merging Large Language Models
por: Goddard, Charles, et al.
Publicado: (2024)

Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment
por: Lu, Keming, et al.
Publicado: (2024)

Tracking Universal Features Through Fine-Tuning and Model Merging
por: Horn, Niels, et al.
Publicado: (2024)

Yuan3.0 Ultra: A Trillion-Parameter Enterprise-Oriented MoE LLM
por: ai, YuanLab., et al.
Publicado: (2026)

Merge-of-Thought Distillation
por: Shen, Zhanming, et al.
Publicado: (2025)

K-Merge: Online Continual Merging of Adapters for On-device Large Language Models
por: Shenaj, Donald, et al.
Publicado: (2025)

WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training
por: Tian, Changxin, et al.
Publicado: (2025)

Adaptive LoRA Merge with Parameter Pruning for Low-Resource Generation
por: Miyano, Ryota, et al.
Publicado: (2025)