Image de couverture de livre

PEMT: Multi-Task Correlation Guided Mixture-of-Experts Enables Parameter-Efficient Transfer Learning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Lin, Zhisheng, Fu, Han, Liu, Chenghao, Li, Zhuo, Sun, Jianling
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques
par: Mao, Qiheng, et autres
Publié: (2024)

MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Models
par: Liu, Zehua, et autres
Publié: (2025)

MoMQ: Mixture-of-Experts Enhances Multi-Dialect Query Generation across Relational and Non-Relational Databases
par: Lin, Zhisheng, et autres
Publié: (2024)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
par: Liu, Yilun, et autres
Publié: (2025)

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
par: Zhang, Zeliang, et autres
Publié: (2024)

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
par: Zheng, Chen, et autres
Publié: (2025)

Less is More: Parameter-Efficient Selection of Intermediate Tasks for Transfer Learning
par: Schulte, David, et autres
Publié: (2024)

MoSLD: An Extremely Parameter-Efficient Mixture-of-Shared LoRAs for Multi-Task Learning
par: Zhao, Lulu, et autres
Publié: (2024)

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
par: Chen, Hao, et autres
Publié: (2026)

S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning
par: Zeng, Hanqing, et autres
Publié: (2025)

CoMoE: Contrastive Representation for Mixture-of-Experts in Parameter-Efficient Fine-tuning
par: Feng, Jinyuan, et autres
Publié: (2025)

Mixture of Lookup Experts
par: Jie, Shibo, et autres
Publié: (2025)

Calibration of Time-Series Forecasting: Detecting and Adapting Context-Driven Distribution Shift
par: Chen, Mouxiang, et autres
Publié: (2023)

ScaLearn: Simple and Highly Parameter-Efficient Task Transfer by Learning to Scale
par: Frohmann, Markus, et autres
Publié: (2023)

Identifiability Matters: Revealing the Hidden Recoverable Condition in Unbiased Learning to Rank
par: Chen, Mouxiang, et autres
Publié: (2023)

Contrastive Learning and Mixture of Experts Enables Precise Vector Embeddings
par: Hallee, Logan, et autres
Publié: (2024)

Beyond instruction-conditioning, MoTE: Mixture of Task Experts for Multi-task Embedding Models
par: Romero, Miguel, et autres
Publié: (2025)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
par: Huang, Wei, et autres
Publié: (2024)

Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts
par: Wang, Haoxiang, et autres
Publié: (2024)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
par: Zhang, Buze, et autres
Publié: (2026)

Maximum Score Routing For Mixture-of-Experts
par: Dong, Bowen, et autres
Publié: (2025)

Towards a Comprehensive Scaling Law of Mixture-of-Experts
par: Zhao, Guoliang, et autres
Publié: (2025)

Routing-Free Mixture-of-Experts
par: Liu, Yilun, et autres
Publié: (2026)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
par: Lu, Xudong, et autres
Publié: (2024)

Beyond Benchmarks: Understanding Mixture-of-Experts Models through Internal Mechanisms
par: Ying, Jiahao, et autres
Publié: (2025)

Multi-Head Mixture-of-Experts
par: Wu, Xun, et autres
Publié: (2024)

Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging
par: Hui, Tingfeng, et autres
Publié: (2024)

Binary-Integer-Programming Based Algorithm for Expert Load Balancing in Mixture-of-Experts Models
par: Sun, Yuan
Publié: (2025)

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
par: Wang, Zihan, et autres
Publié: (2025)

MoPE: Mixture of Prompt Experts for Parameter-Efficient and Scalable Multimodal Fusion
par: Jiang, Ruixiang, et autres
Publié: (2024)

NeuroLoRA: Context-Aware Neuromodulation for Parameter-Efficient Multi-Task Adaptation
par: Yang, Yuxin, et autres
Publié: (2026)

FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Experts
par: Zou, Heming, et autres
Publié: (2025)

MoEMoE: Question Guided Dense and Scalable Sparse Mixture-of-Expert for Multi-source Multi-modal Answering
par: Verma, Vinay Kumar, et autres
Publié: (2025)

HMoE: Heterogeneous Mixture of Experts for Language Modeling
par: Wang, An, et autres
Publié: (2024)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
par: Muzio, Alexandre, et autres
Publié: (2024)

Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts
par: Zhang, Di, et autres
Publié: (2025)

Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
par: Ahrac, Sagi, et autres
Publié: (2026)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
par: Liu, Baihui, et autres
Publié: (2026)

LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning
par: Kowsher, Md, et autres
Publié: (2026)

BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity
par: Song, Chenyang, et autres
Publié: (2025)