Imagen de Portada

MoE-DiffuSeq: Enhancing Long-Document Diffusion Models with Sparse Attention and Mixture of Experts

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Christoforos, Alexandros, Davis, Chadbourne
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

SA-DiffuSeq: Addressing Computational and Scalability Challenges in Long-Document Generation with Sparse Attention
por: Christoforos, Alexandros, et al.
Publicado: (2025)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
por: Muzio, Alexandre, et al.
Publicado: (2024)

S2MoE: Robust Sparse Mixture of Experts via Stochastic Learning
por: Do, Giang, et al.
Publicado: (2025)

MH-MoE: Multi-Head Mixture-of-Experts
por: Huang, Shaohan, et al.
Publicado: (2024)

$\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts
por: Chen, Guanjie, et al.
Publicado: (2024)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
por: Takashiro, Shota, et al.
Publicado: (2026)

LLaDA-MoE: A Sparse MoE Diffusion Language Model
por: Zhu, Fengqi, et al.
Publicado: (2025)

LogicGaze: Benchmarking Causal Consistency in Visual Narratives via Counterfactual Verification
por: Driscoll, Rory, et al.
Publicado: (2026)

DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts
por: Feng, Yuchen, et al.
Publicado: (2025)

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
por: Tang, Yehui, et al.
Publicado: (2025)

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
por: Sun, Weigao, et al.
Publicado: (2025)

Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
por: Jiang, Fan, et al.
Publicado: (2026)

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
por: Pióro, Maciej, et al.
Publicado: (2024)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
por: Wu, Haoyuan, et al.
Publicado: (2025)

PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning
por: Li, Zongqian, et al.
Publicado: (2025)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
por: Gu, Naibin, et al.
Publicado: (2025)

Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts
por: Li, Yunxin, et al.
Publicado: (2024)

Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models
por: Jiang, Songtao, et al.
Publicado: (2024)

Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
por: Wei, Tianwen, et al.
Publicado: (2024)

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
por: Liu, Yang, et al.
Publicado: (2026)

Forgetting as a Feature: Cognitive Alignment of Large Language Models
por: Christoforos, Alexandros
Publicado: (2025)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
por: Qian, Yulei, et al.
Publicado: (2024)

MoE-LPR: Multilingual Extension of Large Language Models through Mixture-of-Experts with Language Priors Routing
por: Zhou, Hao, et al.
Publicado: (2024)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
por: Liu, Baihui, et al.
Publicado: (2026)

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
por: Ludziejewski, Jan, et al.
Publicado: (2025)

$μ$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
por: Koike-Akino, Toshiaki, et al.
Publicado: (2025)

FLAME-MoE: A Transparent End-to-End Research Platform for Mixture-of-Experts Language Models
por: Kang, Hao, et al.
Publicado: (2025)

LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training
por: Zhu, Tong, et al.
Publicado: (2024)

EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
por: Fu, Zhongqian, et al.
Publicado: (2025)

TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Acceleration
por: Wei, Linye, et al.
Publicado: (2026)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
por: Xia, Xinfeng, et al.
Publicado: (2025)

DiffuRank: Effective Document Reranking with Diffusion Language Models
por: Liu, Qi, et al.
Publicado: (2026)

X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms
por: Yuan, Yueming, et al.
Publicado: (2025)

MultiPL-MoE: Multi-Programming-Lingual Extension of Large Language Models through Hybrid Mixture-of-Experts
por: Wang, Qing, et al.
Publicado: (2025)

Expert Selections In MoE Models Reveal (Almost) As Much As Text
por: Nuriyev, Amir, et al.
Publicado: (2026)

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
por: Shi, Jingze, et al.
Publicado: (2026)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
por: Xie, Yanyue, et al.
Publicado: (2024)

Advancing Expert Specialization for Better MoE
por: Guo, Hongcan, et al.
Publicado: (2025)

MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression
por: Sun, Libo, et al.
Publicado: (2026)

Do Domain-specific Experts exist in MoE-based LLMs?
por: Do, Giang, et al.
Publicado: (2026)