Copertina

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Zihan, Chen, Deli, Dai, Damai, Xu, Runxin, Li, Zhuoshu, Wu, Y.
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
di: Dai, Damai, et al.
Pubblicazione: (2024)

ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
di: Xu, Benfeng, et al.
Pubblicazione: (2023)

SciDFM: A Large Language Model with Mixture-of-Experts for Science
di: Sun, Liangtai, et al.
Pubblicazione: (2024)

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
di: Li, Dengchun, et al.
Pubblicazione: (2024)

Routing-Aligned Fine-Tuning for Multilingual Downstream Tasks in Mixture-of-Experts Models
di: Deng, Guanzhi, et al.
Pubblicazione: (2026)

Scaling Sparse Fine-Tuning to Large Language Models
di: Ansell, Alan, et al.
Pubblicazione: (2024)

Geo-Expert: Towards Expert-Level Geological Reasoning via Parameter-Efficient Fine-Tuning
di: Guo, Chenyou, et al.
Pubblicazione: (2026)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
di: Lu, Xudong, et al.
Pubblicazione: (2024)

Pre-Attention Expert Prediction and Prefetching for Mixture-of-Experts Large Language Models
di: Zhu, Shien, et al.
Pubblicazione: (2025)

SparseDoctor: Towards Efficient Chat Doctor with Mixture of Experts Enhanced Large Language Models
di: Zhang, Jianbin, et al.
Pubblicazione: (2025)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
di: Deng, Guanzhi, et al.
Pubblicazione: (2026)

Pruning General Large Language Models into Customized Expert Models
di: Zhao, Yirao, et al.
Pubblicazione: (2025)

Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
di: Wang, Lean, et al.
Pubblicazione: (2024)

Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
di: Wang, Peiyi, et al.
Pubblicazione: (2023)

Hierarchical Alignment: Surgical Fine-Tuning via Functional Layer Specialization in Large Language Models
di: Zhang, Yukun, et al.
Pubblicazione: (2025)

From LLM to Conversational Agent: A Memory Enhanced Architecture with Fine-Tuning of Large Language Models
di: Liu, Na, et al.
Pubblicazione: (2024)

A Study of Large Language Models for Patient Information Extraction: Model Architecture, Fine-Tuning Strategy, and Multi-task Instruction Tuning
di: Peng, Cheng, et al.
Pubblicazione: (2025)

A Multi-Expert Large Language Model Architecture for Verilog Code Generation
di: Nadimi, Bardia, et al.
Pubblicazione: (2024)

Dynamic Reasoning Chains through Depth-Specialized Mixture-of-Experts in Transformer Architectures
di: Roy, Sampurna, et al.
Pubblicazione: (2025)

Upcycling Large Language Models into Mixture of Experts
di: He, Ethan, et al.
Pubblicazione: (2024)

Stochastic Parrots or ICU Experts? Large Language Models in Critical Care Medicine: A Scoping Review
di: Shi, Tongyue, et al.
Pubblicazione: (2024)

HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models
di: Zhang, Wenqiao, et al.
Pubblicazione: (2024)

Large Language Models Can Be a Viable Substitute for Expert Political Surveys When a Shock Disrupts Traditional Measurement Approaches
di: Wu, Patrick Y.
Pubblicazione: (2025)

Impact of Fine-Tuning Methods on Memorization in Large Language Models
di: Hou, Jie, et al.
Pubblicazione: (2025)

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
di: DeepSeek-AI, et al.
Pubblicazione: (2024)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
di: Liu, Yilun, et al.
Pubblicazione: (2025)

Multi-objective Large Language Model Alignment with Hierarchical Experts
di: Li, Zhuo, et al.
Pubblicazione: (2025)

Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models
di: Liu, Xinxin, et al.
Pubblicazione: (2024)

Memorization in Fine-Tuned Large Language Models
di: Savine, Danil
Pubblicazione: (2025)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
di: Zhang, Buze, et al.
Pubblicazione: (2026)

Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
di: Pan, Bowen, et al.
Pubblicazione: (2024)

GigaChat Family: Efficient Russian Language Modeling Through Mixture of Experts Architecture
di: GigaChat team, et al.
Pubblicazione: (2025)

Governance-Aware Hybrid Fine-Tuning for Multilingual Large Language Models
di: Qi, Haomin, et al.
Pubblicazione: (2025)

Training Sparse Mixture Of Experts Text Embedding Models
di: Nussbaum, Zach, et al.
Pubblicazione: (2025)

The New Agronomists: Language Models are Experts in Crop Management
di: Wu, Jing, et al.
Pubblicazione: (2024)

DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
di: Wu, Zhiyu, et al.
Pubblicazione: (2024)

Mix of Experts Language Model for Named Entity Recognition
di: Chen, Xinwei, et al.
Pubblicazione: (2024)

Safety-Aware Fine-Tuning of Large Language Models
di: Choi, Hyeong Kyu, et al.
Pubblicazione: (2024)

Phased Instruction Fine-Tuning for Large Language Models
di: Pang, Wei, et al.
Pubblicazione: (2024)

Channel Merging: Preserving Specialization for Merged Experts
di: Zhang, Mingyang, et al.
Pubblicazione: (2024)