Copertina

PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Zongqian, Su, Yixuan, Collier, Nigel
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

A Survey on Prompt Tuning
di: Li, Zongqian, et al.
Pubblicazione: (2025)

500xCompressor: Generalized Prompt Compression for Large Language Models
di: Li, Zongqian, et al.
Pubblicazione: (2024)

Flexi-LoRA with Input-Adaptive Ranks: Efficient Finetuning for Speech and Reasoning Tasks
di: Li, Zongqian, et al.
Pubblicazione: (2026)

Prompt Compression for Large Language Models: A Survey
di: Li, Zongqian, et al.
Pubblicazione: (2024)

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
di: Tang, Yehui, et al.
Pubblicazione: (2025)

ReasonGraph: Visualisation of Reasoning Paths
di: Li, Zongqian, et al.
Pubblicazione: (2025)

MH-MoE: Multi-Head Mixture-of-Experts
di: Huang, Shaohan, et al.
Pubblicazione: (2024)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
di: Takashiro, Shota, et al.
Pubblicazione: (2026)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
di: Wu, Haoyuan, et al.
Pubblicazione: (2025)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
di: Qian, Yulei, et al.
Pubblicazione: (2024)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
di: Liu, Baihui, et al.
Pubblicazione: (2026)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
di: Muzio, Alexandre, et al.
Pubblicazione: (2024)

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
di: Ludziejewski, Jan, et al.
Pubblicazione: (2025)

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
di: Pióro, Maciej, et al.
Pubblicazione: (2024)

Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
di: Jiang, Fan, et al.
Pubblicazione: (2026)

S2MoE: Robust Sparse Mixture of Experts via Stochastic Learning
di: Do, Giang, et al.
Pubblicazione: (2025)

Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts
di: Li, Yunxin, et al.
Pubblicazione: (2024)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
di: Manzoni, Andrea
Pubblicazione: (2026)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
di: Gu, Naibin, et al.
Pubblicazione: (2025)

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
di: Shi, Jingze, et al.
Pubblicazione: (2026)

COFFEE: A Contrastive Oracle-Free Framework for Event Extraction
di: Zhang, Meiru, et al.
Pubblicazione: (2023)

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
di: Sun, Weigao, et al.
Pubblicazione: (2025)

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
di: Liu, Yang, et al.
Pubblicazione: (2026)

Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models
di: Li, Zongqian, et al.
Pubblicazione: (2026)

Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models
di: Jiang, Songtao, et al.
Pubblicazione: (2024)

Unlocking Structure Measuring: Introducing PDD, an Automatic Metric for Positional Discourse Coherence
di: Liu, Yinhong, et al.
Pubblicazione: (2024)

$μ$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
di: Koike-Akino, Toshiaki, et al.
Pubblicazione: (2025)

MoLEx: Mixture of Layer Experts for Finetuning with Sparse Upcycling
di: Teo, Rachel S. Y., et al.
Pubblicazione: (2025)

Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
di: Wei, Tianwen, et al.
Pubblicazione: (2024)

DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts
di: Feng, Yuchen, et al.
Pubblicazione: (2025)

MoE-DiffuSeq: Enhancing Long-Document Diffusion Models with Sparse Attention and Mixture of Experts
di: Christoforos, Alexandros, et al.
Pubblicazione: (2025)

LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training
di: Zhu, Tong, et al.
Pubblicazione: (2024)

X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms
di: Yuan, Yueming, et al.
Pubblicazione: (2025)

$\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts
di: Chen, Guanjie, et al.
Pubblicazione: (2024)

MoE-LPR: Multilingual Extension of Large Language Models through Mixture-of-Experts with Language Priors Routing
di: Zhou, Hao, et al.
Pubblicazione: (2024)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
di: Xia, Xinfeng, et al.
Pubblicazione: (2025)

FLAME-MoE: A Transparent End-to-End Research Platform for Mixture-of-Experts Language Models
di: Kang, Hao, et al.
Pubblicazione: (2025)

Attention Instruction: Amplifying Attention in the Middle via Prompting
di: Zhang, Meiru, et al.
Pubblicazione: (2024)

BLR-MoE: Boosted Language-Routing Mixture of Experts for Domain-Robust Multilingual E2E ASR
di: Ma, Guodong, et al.
Pubblicazione: (2025)

MoMoE: Mixture of Moderation Experts Framework for AI-Assisted Online Governance
di: Goyal, Agam, et al.
Pubblicazione: (2025)