Cover Image

One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Le, Minh, Dao, Bao-Ngoc, Nguyen, Huy, Tran, Quyen, Nguyen, Anh, Ho, Nhat
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Revisit Visual Prompt Tuning: The Expressiveness of Prompt Experts
by: Le, Minh, et al.
Published: (2025)

Mixture of Experts Meets Prompt-Based Continual Learning
by: Le, Minh, et al.
Published: (2024)

Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts
by: Le, Minh, et al.
Published: (2024)

Attack On Prompt: Backdoor Attack in Prompt-Based Continual Learning
by: Nguyen, Trang, et al.
Published: (2024)

Leveraging Hierarchical Taxonomies in Prompt-based Continual Learning
by: Tran, Quyen, et al.
Published: (2024)

On Parameter Estimation in Deviated Gaussian Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2024)

RepLoRA: Reparameterizing Low-Rank Adaptation via the Perspective of Mixture of Experts
by: Truong, Tuan, et al.
Published: (2025)

Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2023)

Convergence Rates for Softmax Gating Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2025)

On Least Square Estimation in Softmax Gating Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2024)

Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts?
by: Nguyen, Huy, et al.
Published: (2024)

Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2023)

Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2024)

Rethinking Multinomial Logistic Mixture of Experts with Sigmoid Gating Function
by: Pham, Tuan Minh, et al.
Published: (2026)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
by: Nguyen-Nhat, Minh-Khoi, et al.
Published: (2025)

Expert Merging in Sparse Mixture of Experts with Nash Bargaining
by: Nguyen, Dung V., et al.
Published: (2025)

A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2023)

Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts
by: Yan, Fanqi, et al.
Published: (2024)

Quadratic Gating Mixture of Experts: Statistical Insights into Self-Attention
by: Akbarian, Pedram, et al.
Published: (2024)

Improving Routing in Sparse Mixture of Experts with Graph of Tokens
by: Nguyen, Tam, et al.
Published: (2025)

Improving Minimax Estimation Rates for Contaminated Mixture of Multinomial Logistic Experts via Expert Heterogeneity
by: Yan, Fanqi, et al.
Published: (2026)

On Zero-Initialized Attention: Optimal Prompt and Gating Factor Estimation
by: Diep, Nghiem T., et al.
Published: (2025)

On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions
by: Nguyen, Huy, et al.
Published: (2024)

Selective Sinkhorn Routing for Improved Sparse Mixture of Experts
by: Nguyen, Duc Anh, et al.
Published: (2025)

Statistical Advantages of Perturbing Cosine Router in Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2024)

On Minimax Estimation of Parameters in Softmax-Contaminated Mixture of Experts
by: Yan, Fanqi, et al.
Published: (2025)

A Statistical Theory of Gated Attention through the Lens of Hierarchical Mixture of Experts
by: Nguyen, Viet, et al.
Published: (2026)

FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion
by: Han, Xing, et al.
Published: (2024)

On Bayesian Softmax-Gated Mixture-of-Experts Models
by: Bariletto, Nicola, et al.
Published: (2026)

CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition
by: Pham, Quang, et al.
Published: (2024)

LEAF: A Robust Expert-Based Framework for Few-Shot Continual Event Detection
by: Dao, Bao-Ngoc, et al.
Published: (2025)

KOPPA: Improving Prompt-based Continual Learning with Key-Query Orthogonal Projection and Prototype-based One-Versus-All
by: Tran, Quyen, et al.
Published: (2023)

Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective
by: Le, Minh, et al.
Published: (2024)

Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspective
by: Yan, Fanqi, et al.
Published: (2025)

Wasserstein Gaussianization and Efficient Variational Bayes for Robust Bayesian Synthetic Likelihood
by: Nguyen, Nhat-Minh, et al.
Published: (2023)

Adaptive multi-gradient methods for quasiconvex vector optimization and applications to multi-task learning
by: Minh, Nguyen Anh, et al.
Published: (2024)

Improving Generalization with Flat Hilbert Bayesian Inference
by: Truong, Tuan, et al.
Published: (2024)

Enhancing Domain Adaptation through Prompt Gradient Alignment
by: Phan, Hoang, et al.
Published: (2024)

MP-MoE: Matrix Profile-Guided Mixture of Experts for Precipitation Forecasting
by: Tran, Huyen Ngoc, et al.
Published: (2026)

MGPATH: Vision-Language Model with Multi-Granular Prompt Learning for Few-Shot WSI Classification
by: Nguyen, Anh-Tien, et al.
Published: (2025)