Cover Image

S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zeng, Hanqing, Xia, Yinglong, Zhao, Zhuokai, Jiang, Chuan, Zhang, Qiang, Liu, Jiayi, Zhang, Qunshu, Zhang, Lizhu, Fan, Xiangjun, Zhang, Benyu
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts
by: Feng, Jiarui, et al.
Published: (2026)

TARo: Token-level Adaptive Routing for LLM Test-time Alignment
by: Rai, Arushi, et al.
Published: (2026)

Mixture of Weak & Strong Experts on Graphs
by: Zeng, Hanqing, et al.
Published: (2023)

GEM: Empowering LLM for both Embedding Generation and Language Understanding
by: Zhang, Caojin, et al.
Published: (2025)

RecoWorld: Building Simulated Environments for Agentic Recommender Systems
by: Liu, Fei, et al.
Published: (2025)

Synthetic Sandbox for Training Machine Learning Engineering Agents
by: Zhou, Yuhang, et al.
Published: (2026)

CoMoE: Contrastive Representation for Mixture-of-Experts in Parameter-Efficient Fine-tuning
by: Feng, Jinyuan, et al.
Published: (2025)

Agentic Recommender System with Hierarchical Belief-State Memory
by: Shen, Xiang, et al.
Published: (2026)

Token-Level LLM Collaboration via FusionRoute
by: Xiong, Nuoya, et al.
Published: (2026)

OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification
by: Zhou, Yuhang, et al.
Published: (2026)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
by: Zhang, Buze, et al.
Published: (2026)

Efficient Sequential Recommendation for Long Term User Interest Via Personalization
by: Zhang, Qiang, et al.
Published: (2026)

ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning
by: Qiu, Ruizhong, et al.
Published: (2026)

MoRE: A Mixture of Low-Rank Experts for Adaptive Multi-Task Learning
by: Zhang, Dacao, et al.
Published: (2025)

MoRE: Mixture of Residual Experts for Humanoid Lifelike Gaits Learning on Complex Terrains
by: Wang, Dewei, et al.
Published: (2025)

Don't Waste It: Guiding Generative Recommenders with Structured Human Priors via Multi-Head Decoding
by: Zhang, Yunkai, et al.
Published: (2025)

Exploring System 1 and 2 communication for latent reasoning in LLMs
by: Coda-Forno, Julian, et al.
Published: (2025)

Mixture-of-Minds: Multi-Agent Reinforcement Learning for Table Understanding
by: Zhou, Yuhang, et al.
Published: (2025)

CAFe: Unifying Representation and Generation with Contrastive-Autoregressive Finetuning
by: Yu, Hao, et al.
Published: (2025)

ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models
by: Singh, Gursimran, et al.
Published: (2025)

Principled Synthetic Data Enables the First Scaling Laws for LLMs in Recommendation
by: Zhang, Benyu, et al.
Published: (2026)

MoRE-LLM: Mixture of Rule Experts Guided by a Large Language Model
by: Koebler, Alexander, et al.
Published: (2025)

MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Models
by: Cao, Jie, et al.
Published: (2025)

Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment
by: Wang, Chaoqi, et al.
Published: (2025)

Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning
by: Xu, Jing, et al.
Published: (2024)

MoDE: Effective Multi-task Parameter Efficient Fine-Tuning with a Mixture of Dyadic Experts
by: Ning, Lin, et al.
Published: (2024)

TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
by: Kunwar, Pradip, et al.
Published: (2025)

DiaMoE-TTS: A Unified IPA-Based Dialect TTS Framework with Mixture-of-Experts and Parameter-Efficient Zero-Shot Adaptation
by: Chen, Ziqi, et al.
Published: (2025)

FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation
by: Zhan, Ziwei, et al.
Published: (2024)

MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Models
by: Liu, Zehua, et al.
Published: (2025)

MoRE-Brain: Routed Mixture of Experts for Interpretable and Generalizable Cross-Subject fMRI Visual Decoding
by: Wei, Yuxiang, et al.
Published: (2025)

MoC-System: Efficient Fault Tolerance for Sparse Mixture-of-Experts Model Training
by: Cai, Weilin, et al.
Published: (2024)

FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing
by: Zhang, Boyang, et al.
Published: (2025)

AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models
by: Zeng, Zihao, et al.
Published: (2024)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
by: Liu, Yilun, et al.
Published: (2025)

Mixture of Physical Priors Adapter for Parameter-Efficient Fine-Tuning
by: Wang, Zhaozhi, et al.
Published: (2024)

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
by: Liu, Yilun, et al.
Published: (2024)

MLP Fusion: Towards Efficient Fine-tuning of Dense and Mixture-of-Experts Language Models
by: Ai, Mengting, et al.
Published: (2023)

Training Diverse Graph Experts for Ensembles: A Systematic Empirical Study
by: Deng, Gangda, et al.
Published: (2025)

MoRE: 3D Visual Geometry Reconstruction Meets Mixture-of-Experts
by: Gao, Jingnan, et al.
Published: (2025)