Cover Image

MossNet: Mixture of State-Space Experts is a Multi-Head Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tuli, Shikhar, Smith, James Seale, Jeelani, Haris, Lin, Chi-Heng, Patel, Abhishek, Ramanishka, Vasili, Hsu, Yen-Chang, Jin, Hongxia
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FlexiGPT: Pruning and Extending Large Language Models with Low-Rank Weight Sharing
by: Smith, James Seale, et al.
Published: (2025)

MoDeGPT: Modular Decomposition for Large Language Model Compression
by: Lin, Chi-Heng, et al.
Published: (2024)

DynaMo: Accelerating Language Model Inference with Dynamic Multi-Token Sampling
by: Tuli, Shikhar, et al.
Published: (2024)

Continual Diffusion with STAMINA: STack-And-Mask INcremental Adapters
by: Smith, James Seale, et al.
Published: (2023)

Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA
by: Smith, James Seale, et al.
Published: (2023)

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
by: Csordás, Róbert, et al.
Published: (2023)

Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts
by: Li, Cheng, et al.
Published: (2025)

Multi-Head Mixture-of-Experts
by: Wu, Xun, et al.
Published: (2024)

Modeling Missing at Random Neuropsychological Test Scores Using a Mixture of Binomial Product Experts
by: Suen, Daniel, et al.
Published: (2023)

GraphMERT: Efficient and Scalable Distillation of Reliable Knowledge Graphs from Unstructured Data
by: Belova, Margarita, et al.
Published: (2025)

Unifying Mixture of Experts and Multi-Head Latent Attention for Efficient Language Models
by: Mehta, Sushant, et al.
Published: (2025)

Focus Where It Matters: Graph Selective State Focused Attention Networks
by: Vashistha, Shikhar, et al.
Published: (2024)

Linear Predictability of Attention Heads in Large Language Models
by: Shaikh, Khalid, et al.
Published: (2026)

DISP-LLM: Dimension-Independent Structural Pruning for Large Language Models
by: Gao, Shangqian, et al.
Published: (2024)

MoH: Multi-Head Attention as Mixture-of-Head Attention
by: Jin, Peng, et al.
Published: (2024)

MambaCSP: Hybrid-Attention State Space Models for Hardware-Efficient Channel State Prediction
by: Djuhera, Aladin, et al.
Published: (2026)

ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning
by: Gao, Shangqian, et al.
Published: (2025)

BlackMamba: Mixture of Experts for State-Space Models
by: Anthony, Quentin, et al.
Published: (2024)

Netting in Black Moss Tarn 1954
Published: (1954)

Towards 3D Acceleration for low-power Mixture-of-Experts and Multi-Head Attention Spiking Transformers
by: Xu, Boxun, et al.
Published: (2024)

SHMoAReg: Spark Deformable Image Registration via Spatial Heterogeneous Mixture of Experts and Attention Heads
by: Zheng, Yuxi, et al.
Published: (2025)

MH-MoE: Multi-Head Mixture-of-Experts
by: Huang, Shaohan, et al.
Published: (2024)

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
by: Zhan, Zheng, et al.
Published: (2025)

PathFinder: MCTS and LLM Feedback-based Path Selection for Multi-Hop Question Answering
by: Maram, Durga Prasad, et al.
Published: (2025)

Let the Experts Speak: Improving Survival Prediction & Calibration via Mixture-of-Experts Heads
by: Morrill, Todd, et al.
Published: (2025)

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
by: Pióro, Maciej, et al.
Published: (2024)

What is the most optimal diffusion?
by: Baranau, Vasili
Published: (2025)

El Lenin oculto
by: Grossman, Vasili
Published: (1990)

Facet-Aware Multi-Head Mixture-of-Experts Model for Sequential Recommendation
by: Liu, Mingrui, et al.
Published: (2024)

Context-Aware Dialectal Arabic Machine Translation with Interactive Region and Register Selection
by: Nowshin, Afroza, et al.
Published: (2026)

Pre-Attention Expert Prediction and Prefetching for Mixture-of-Experts Large Language Models
by: Zhu, Shien, et al.
Published: (2025)

EEGMamba: Bidirectional State Space Model with Mixture of Experts for EEG Multi-task Classification
by: Gui, Yiyu, et al.
Published: (2024)

Pollen profile SLUGGAN, Sluggan Moss, Ireland
by: Smith, A G
Published: (2010)

MTmixAtt: Integrating Mixture-of-Experts with Multi-Mix Attention for Large-Scale Recommendation
by: Qi, Xianyang, et al.
Published: (2025)

Simultaneous Determination of 10 Fat‐Soluble Vitamins by Ultra‐Performance Liquid Chromatography in Multivitamins With Minerals Capsules
by: Salika Jeelani, et al.
Published: (2025)

Unveiling Reasoning Thresholds in Language Models: Scaling, Fine-Tuning, and Interpretability through Attention Maps
by: Hsiao, Yen-Che, et al.
Published: (2025)

Marketing Information Literacy
by: Seale, Maura
Published: (2013)

Mexican-American Folklore: An Approach to the Research Paper.
by: Seale, Jan
Published: (1976)

Critical Library Instruction, Causing Trouble, and Institutionalization
by: Seale, Maura
Published: (2020)

ANALYSIS OF HEALTH COMMUNICATION TEXTS UK PRESS COVERAGE OF DEBATES ABOUT ASSISTED DYING
by: Clive Seale
Published: (2016)