Copertina

Routing-Aligned Fine-Tuning for Multilingual Downstream Tasks in Mixture-of-Experts Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Deng, Guanzhi, Wu, Kuan, Wang, Haibo, Wong, Shing Yin, Luo, Sichun, Song, Linqi
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Can LLM Substitute Human Labeling? A Case Study of Fine-grained Chinese Address Entity Recognition Dataset for UAV Delivery
di: Yao, Yuxuan, et al.
Pubblicazione: (2024)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
di: Deng, Guanzhi, et al.
Pubblicazione: (2026)

Multilingual Routing in Mixture-of-Experts
di: Bandarkar, Lucas, et al.
Pubblicazione: (2025)

Enhancing Low-Rank Adaptation with Structured Nonlinear Transformations
di: Deng, Guanzhi, et al.
Pubblicazione: (2025)

Towards Improving Interpretability of Language Model Generation through a Structured Knowledge Discovery Approach
di: Liu, Shuqi, et al.
Pubblicazione: (2025)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
di: Liu, Yilun, et al.
Pubblicazione: (2025)

DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanism
di: Li, Dengchun, et al.
Pubblicazione: (2025)

Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model
di: Li, Chong, et al.
Pubblicazione: (2025)

Aligning Multilingual Reasoning with Verifiable Semantics from a High-Resource Expert Model
di: Faisal, Fahim, et al.
Pubblicazione: (2025)

Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensembling
di: Yao, Yuxuan, et al.
Pubblicazione: (2024)

Reasoning Meets Personalization: Unleashing the Potential of Large Reasoning Model for Personalized Generation
di: Luo, Sichun, et al.
Pubblicazione: (2025)

Probing Semantic Routing in Large Mixture-of-Expert Models
di: Olson, Matthew Lyle, et al.
Pubblicazione: (2025)

Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning
di: Huang, Wenke, et al.
Pubblicazione: (2024)

Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
di: Jiang, Fan, et al.
Pubblicazione: (2026)

Routing-Free Mixture-of-Experts
di: Liu, Yilun, et al.
Pubblicazione: (2026)

Routing by Analogy: kNN-Augmented Expert Assignment for Mixture-of-Experts
di: Lyu, Boxuan, et al.
Pubblicazione: (2026)

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
di: Li, Dengchun, et al.
Pubblicazione: (2024)

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
di: Liu, Yilun, et al.
Pubblicazione: (2024)

Cosine-Similarity Routing with Semantic Anchors for Interpretable Mixture-of-Experts Language Models
di: Ternovtsii, Ivan, et al.
Pubblicazione: (2025)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
di: Zhang, Buze, et al.
Pubblicazione: (2026)

Tuning Language Models by Mixture-of-Depths Ensemble
di: Luo, Haoyan, et al.
Pubblicazione: (2024)

AnyTaskTune: Advanced Domain-Specific Solutions through Task-Fine-Tuning
di: Cui, Jiaxi, et al.
Pubblicazione: (2024)

The Real, the Better: Aligning Large Language Models with Online Human Behaviors
di: Jiang, Guanying, et al.
Pubblicazione: (2024)

KnowTuning: Knowledge-aware Fine-tuning for Large Language Models
di: Lyu, Yougang, et al.
Pubblicazione: (2024)

Multi-domain Knowledge Graph Collaborative Pre-training and Prompt Tuning for Diverse Downstream Tasks
di: Zhang, Yichi, et al.
Pubblicazione: (2024)

SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution
di: Luo, Sichun, et al.
Pubblicazione: (2026)

Fine-Tuning or Fine-Failing? Debunking Performance Myths in Large Language Models
di: Barnett, Scott, et al.
Pubblicazione: (2024)

LD-MoLE: Learnable Dynamic Routing for Mixture of LoRA Experts
di: Zhuang, Yuan, et al.
Pubblicazione: (2025)

Scaling Laws for Fine-Grained Mixture of Experts
di: Krajewski, Jakub, et al.
Pubblicazione: (2024)

Impact of Fine-Tuning Methods on Memorization in Large Language Models
di: Hou, Jie, et al.
Pubblicazione: (2025)

Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning Tasks
di: Nakamura, Taishi, et al.
Pubblicazione: (2025)

Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model
di: Huang, Wenke, et al.
Pubblicazione: (2025)

Optimising Language Models for Downstream Tasks: A Post-Training Perspective
di: Shi, Zhengyan
Pubblicazione: (2025)

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models
di: Wang, Zihan, et al.
Pubblicazione: (2024)

CORAL: Adaptive Retrieval Loop for Culturally-Aligned Multilingual RAG
di: Lee, Nayeon, et al.
Pubblicazione: (2026)

Taiyi: A Bilingual Fine-Tuned Large Language Model for Diverse Biomedical Tasks
di: Luo, Ling, et al.
Pubblicazione: (2023)

Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts
di: Xu, Haolei, et al.
Pubblicazione: (2026)

Geo-Expert: Towards Expert-Level Geological Reasoning via Parameter-Efficient Fine-Tuning
di: Guo, Chenyou, et al.
Pubblicazione: (2026)

How to Tune a Multilingual Encoder Model for Germanic Languages: A Study of PEFT, Full Fine-Tuning, and Language Adapters
di: Oji, Romina, et al.
Pubblicazione: (2025)

Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging
di: Hui, Tingfeng, et al.
Pubblicazione: (2024)