Image de couverture de livre

State-of-the-Art Arabic Language Modeling with Sparse MoE Fine-Tuning and Chain-of-Thought Distillation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Singh, Navan Preet, Garikipati, Anurag, Abulkhair, Ahmed, Jagdishbhai, Jyani Akshay, Yaduvanshi, Atul, Chaudhary, Amarendra, Ciobanu, Madalina, Mao, Qingqing, Das, Ritankar
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Application-Driven Pedagogical Knowledge Optimization of Open-Source LLMs via Reinforcement Learning and Supervised Fine-Tuning
par: Singh, Navan Preet, et autres
Publié: (2026)

OpenMedLM: Prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language models
par: Maharjan, Jenish, et autres
Publié: (2024)

AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates
par: Chen, Shaolong, et autres
Publié: (2026)

Therapeutic Horizons in Targeting EZH2 With Dual and Non‐PROTAC Inhibitor Molecules: Recent Achievements, Comparative Analysis, and Future Perspectives
par: Hamada S. Abulkhair
Publié: (2026)

LLaDA-MoE: A Sparse MoE Diffusion Language Model
par: Zhu, Fengqi, et autres
Publié: (2025)

Domain Of Supply Chain Management - A State Of Art
par: Atul B. Borade
Publié: (2007)

FFT-MoE: Efficient Federated Fine-Tuning for Foundation Models via Large-scale Sparse MoE under Heterogeneous Edge
par: Hu, Gang, et autres
Publié: (2025)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
par: Manzoni, Andrea
Publié: (2026)

Slow Tuning and Low-Entropy Masking for Safe Chain-of-Thought Distillation
par: Ma, Ziyang, et autres
Publié: (2025)

Study on Benthic organisms of Ghazal Ozan River in Zanjan Province
par: Navan Maghsoodi, M.
Publié: (2013)

Nucleus-Image: Sparse MoE for Image Generation
par: Akiti, Chandan, et autres
Publié: (2026)

Sparse Crosscoders for diffing MoEs and Dense models
par: Chaudhari, Marmik, et autres
Publié: (2026)

Sparse Checkpointing for Fast and Reliable MoE Training
par: Gandhi, Swapnil, et autres
Publié: (2024)

TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
par: Kunwar, Pradip, et autres
Publié: (2025)

MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE
par: Huang, Zongle, et autres
Publié: (2025)

Thomason's completion for K-theory and cyclic homology of quotient stacks
par: Krishna, Amalendu, et autres
Publié: (2025)

GRIN: GRadient-INformed MoE
par: Liu, Liyuan, et autres
Publié: (2024)

Retraining-Free Merging of Sparse MoE via Hierarchical Clustering
par: Chen, I-Chun, et autres
Publié: (2024)

EvoESAP: Non-Uniform Expert Pruning for Sparse MoE
par: Liu, Zongfang, et autres
Publié: (2026)

How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding
par: Chen, Xi, et autres
Publié: (2025)

Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
par: Li, Pingzhi, et autres
Publié: (2025)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
par: Li, Zichong, et autres
Publié: (2025)

LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillation
par: Shu, Fangxun, et autres
Publié: (2024)

A decade of research effort in synthesis, biological activity assessments, and mechanistic investigations of sulfamethazine‐incorporating molecules
par: Hamada S. Abulkhair, et autres
Publié: (2025)

P-CRE-DML: A Novel Approach for Causal Inference in Non-Linear Panel Data
par: Sharma, Amarendra
Publié: (2025)

The Quantum Reserve Token: A Decentralized Digital Currency Backed by Quantum Computational Capacity as a Candidate for Global Reserve Status
par: Sharma, Amarendra
Publié: (2025)

Trade and industrial policies in the developing countries of East Asia / Amarendra Bhattacharya, Johannes F. Linn
par: Bhattacharya, Amarendra
Publié: (1988)

Where to find the beginning of time
par: Swarup, Amarendra
Publié: (2005)

We know zilch about "hot" superconductors
par: Swarup, Amarendra
Publié: (2005)

Quantum-Inspired Cournot Model
par: Sharma, Amarendra
Publié: (2025)

Academic Research Output Derivatives: Structuring Futures and Options on Research Output Index
par: Sharma, Amarendra
Publié: (2025)

Does a Global Perspective Help Prune Sparse MoEs Elegantly?
par: Zhang, Zeliang, et autres
Publié: (2026)

Traffic-MoE: A Sparse Foundation Model for Network Traffic Analysis
par: Zhou, Jiajun, et autres
Publié: (2026)

S2MoE: Robust Sparse Mixture of Experts via Stochastic Learning
par: Do, Giang, et autres
Publié: (2025)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
par: Muzio, Alexandre, et autres
Publié: (2024)

Temporal Chain of Thought: Long-Video Understanding by Thinking in Frames
par: Arnab, Anurag, et autres
Publié: (2025)

When Does Sparse MoE Help in Vision? The Role of Backbone Compute Leverage in Sparse Routing
par: Sun, Libo, et autres
Publié: (2026)

Mind the Gap: Bridging Thought Leap for Improved Chain-of-Thought Tuning
par: Xu, Haolei, et autres
Publié: (2025)

On the Impact of Fine-Tuning on Chain-of-Thought Reasoning
par: Lobo, Elita, et autres
Publié: (2024)

MoE-Prefill: Zero Redundancy Overheads in MoE Prefill Serving
par: Su, Zhaoyuan, et autres
Publié: (2026)