Cover Image

Retraining-Free Merging of Sparse MoE via Hierarchical Clustering

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, I-Chun, Liu, Hsu-Shen, Sun, Wei-Fang, Chao, Chen-Hao, Hsu, Yen-Chang, Lee, Chun-Yi
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
by: Miao, Ruijie, et al.
Published: (2025)

Is Retraining-Free Enough? The Necessity of Router Calibration for Efficient MoE Compression
by: Hyeon, Sieun, et al.
Published: (2026)

Incorporating Damped Harmonic Oscillator in DSGE Models
by: Hsu, Wei Chun
Published: (2025)

EvoESAP: Non-Uniform Expert Pruning for Sparse MoE
by: Liu, Zongfang, et al.
Published: (2026)

LLaDA-MoE: A Sparse MoE Diffusion Language Model
by: Zhu, Fengqi, et al.
Published: (2025)

Efficient CPU-GPU Collaborative Inference for MoE-based LLMs on Memory-Limited Systems
by: Huang, En-Ming, et al.
Published: (2025)

OVOR: OnePrompt with Virtual Outlier Regularization for Rehearsal-Free Class-Incremental Learning
by: Huang, Wei-Cheng, et al.
Published: (2024)

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
by: Li, Lujun, et al.
Published: (2025)

Successful treatment of a mixed candidal and bacterial skull base osteomyelitis with antimicrobial agents and hyperbaric oxygen therapy—A rare case report
by: Yu‐Hsin Liu, et al.
Published: (2025)

Enhancing Function-Calling Capabilities in LLMs: Strategies for Prompt Formats, Data Integration, and Multilingual Translation
by: Chen, Yi-Chang, et al.
Published: (2024)

OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
by: Wang, Liujianfu, et al.
Published: (2025)

Monkey Jump : MoE-Style PEFT for Efficient Multi-Task Learning
by: Prottasha, Nusrat Jahan, et al.
Published: (2026)

MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs
by: Zhou, Yuhang, et al.
Published: (2025)

Traffic-MoE: A Sparse Foundation Model for Network Traffic Analysis
by: Zhou, Jiajun, et al.
Published: (2026)

Efficient estimation for flexible spatial zero-inflated models with environmental applications
by: Shen, Chung-Wei, et al.
Published: (2025)

Cool-Core Destruction in Merging Clusters with AGN Feedback and Radiative Cooling
by: Chen, Shuang-Shuang, et al.
Published: (2024)

SceneFoundry: Generating Interactive Infinite 3D Worlds
by: Chen, ChunTeng, et al.
Published: (2026)

Robust Symbolic Reasoning for Visual Narratives via Hierarchical and Semantically Normalized Knowledge Graphs
by: Chen, Yi-Chun
Published: (2025)

AIMER: Calibration-Free Task-Agnostic MoE Pruning
by: Liu, Zongfang, et al.
Published: (2026)

Delta Decompression for MoE-based LLMs Compression
by: Gu, Hao, et al.
Published: (2025)

Geometrization of summation formulae for quadrics
by: Hsu, Chun-Hsien
Published: (2026)

Asymptotics of Schwartz functions
by: Hsu, Chun-Hsien
Published: (2021)

Anomaly Reparametrization of the Ligon--Schaaf Regularization in the Kepler problem
by: Hsu, Li-Chun
Published: (2026)

GazeFormer-MoE: Context-Aware Gaze Estimation via CLIP and MoE Transformer
by: Zhao, Xinyuan, et al.
Published: (2026)

Nucleus-Image: Sparse MoE for Image Generation
by: Akiti, Chandan, et al.
Published: (2026)

GRIN: GRadient-INformed MoE
by: Liu, Liyuan, et al.
Published: (2024)

Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
by: Chao, Chen-Hao, et al.
Published: (2025)

Unusual rectal polyp
by: Chun‐Chi Hsu, et al.
Published: (2024)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
by: Wu, Haoyuan, et al.
Published: (2025)

DriveEnv-NeRF: Exploration of A NeRF-Based Autonomous Driving Environment for Real-World Performance Validation
by: Shen, Mu-Yi, et al.
Published: (2024)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
by: Xia, Xinfeng, et al.
Published: (2025)

Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Search
by: Liu, Max, et al.
Published: (2024)

Effective Manipulation of Water Droplets on Open Superhydrophobic Glass Surfaces by Using a Triboelectrically Charged Polytetrafluoroethylene Rod on the Back Side of These Surfaces
by: Wei Chen Huang, et al.
Published: (2025)

Effective Manipulation of Water Droplets on Open Superhydrophobic Glass Surfaces by Using a Triboelectrically Charged Polytetrafluoroethylene Rod on the Back Side of These Surfaces (Adv. Mater. Interfaces 12/2025)
by: Wei Chen Huang, et al.
Published: (2025)

Hierarchical Knowledge Graphs for Story Understanding in Visual Narratives
by: Chen, Yi-Chun
Published: (2025)

Hierarchical LoRA MoE for Efficient CTR Model Scaling
by: Zeng, Zhichen, et al.
Published: (2025)

BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generation
by: Pai, Tsung-Min, et al.
Published: (2025)

Maximum Entropy Reinforcement Learning via Energy-Based Normalizing Flow
by: Chao, Chen-Hao, et al.
Published: (2024)

Psychometric evaluation of the Chinese modified perceived neighborhood cohesion scale in older adults at high risk of cognitive impairment
by: Yi‐Chen Chiu, et al.
Published: (2024)

SecMoE: Communication-Efficient Secure MoE Inference via Select-Then-Compute
by: Shen, Bowen, et al.
Published: (2026)