Image de couverture de livre

M$^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Mishra, Mayank, Tan, Shawn, Stoica, Ion, Gonzalez, Joseph, Dao, Tri
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations
par: Guo, Wentao, et autres
Publié: (2025)

Mamba: Linear-Time Sequence Modeling with Selective State Spaces
par: Gu, Albert, et autres
Publié: (2023)

HadamRNN: Binary and Sparse Ternary Orthogonal RNNs
par: Foucault, Armand, et autres
Publié: (2025)

Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers
par: Hwang, Sukjun, et autres
Publié: (2024)

FlashRNN: I/O-Aware Optimization of Traditional RNNs on modern hardware
par: Pöppel, Korbinian, et autres
Publié: (2024)

$\texttt{lrnnx}$: A library for Linear RNNs
par: Bania, Karan, et autres
Publié: (2026)

Fairness in Serving Large Language Models
par: Sheng, Ying, et autres
Publié: (2023)

Implicit Language Models are RNNs: Balancing Parallelization and Expressivity
par: Schöne, Mark, et autres
Publié: (2025)

Post-Training Sparse Attention with Double Sparsity
par: Yang, Shuo, et autres
Publié: (2024)

HashAttention: Semantic Sparsity for Faster Inference
par: Desai, Aditya, et autres
Publié: (2024)

SLA2: Sparse-Linear Attention with Learnable Routing and QAT
par: Zhang, Jintao, et autres
Publié: (2026)

BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation
par: Zhu, Alan, et autres
Publié: (2025)

Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
par: Park, Jongseok, et autres
Publié: (2026)

MELCOT: A Hybrid Learning Architecture with Marginal Preservation for Matrix-Valued Regression
par: Tran, Khang, et autres
Publié: (2025)

Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernels
par: Beck, Maximilian, et autres
Publié: (2025)

K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model
par: Cao, Shiyi, et autres
Publié: (2026)

S*: Test Time Scaling for Code Generation
par: Li, Dacheng, et autres
Publié: (2025)

SKOLR: Structured Koopman Operator Linear RNN for Time-Series Forecasting
par: Zhang, Yitian, et autres
Publié: (2025)

A Statistical Framework for Ranking LLM-Based Chatbots
par: Ameli, Siavash, et autres
Publié: (2024)

Linear RNNs for autoregressive generation of long music samples
par: Szewczyk, Konrad, et autres
Publié: (2025)

TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting
par: Moroshan, Vladyslav, et autres
Publié: (2025)

SageBwd: A Trainable Low-bit Attention
par: Zhang, Jintao, et autres
Publié: (2026)

Memory Caching: RNNs with Growing Memory
par: Behrouz, Ali, et autres
Publié: (2026)

Were RNNs All We Needed?
par: Feng, Leo, et autres
Publié: (2024)

Learning to (Learn at Test Time): RNNs with Expressive Hidden States
par: Sun, Yu, et autres
Publié: (2024)

The Mamba in the Llama: Distilling and Accelerating Hybrid Models
par: Wang, Junxiong, et autres
Publié: (2024)

PIAD-SRNN: Physics-Informed Adaptive Decomposition in State-Space RNN
par: Mohammadshirazi, Ahmad, et autres
Publié: (2024)

Inference Time Context Sparsity: Illusion or Opportunity?
par: Joshi, Sahil, et autres
Publié: (2026)

Pessimistic Iterative Planning with RNNs for Robust POMDPs
par: Galesloot, Maris F. L., et autres
Publié: (2024)

vAttention: Verified Sparse Attention
par: Desai, Aditya, et autres
Publié: (2025)

Trustless Audits without Revealing Data or Models
par: Waiwitlikhit, Suppakit, et autres
Publié: (2024)

Uncovering the Computational Roles of Nonlinearity in Sequence Modeling Using Almost-Linear RNNs
par: Brenner, Manuel, et autres
Publié: (2025)

TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times
par: Zhang, Jintao, et autres
Publié: (2025)

Balancing Sparse RNNs with Hyperparameterization Benefiting Meta-Learning
par: Hershey, Quincy, et autres
Publié: (2025)

StateX: Enhancing RNN Recall via Post-training State Expansion
par: Shen, Xingyu, et autres
Publié: (2025)

How to Evaluate Reward Models for RLHF
par: Frick, Evan, et autres
Publié: (2024)

MPC-Minimized Secure LLM Inference
par: Rathee, Deevashwer, et autres
Publié: (2024)

Does Transformer Interpretability Transfer to RNNs?
par: Paulo, Gonçalo, et autres
Publié: (2024)

Paradoxical noise preference in RNNs
par: Eckstein, Noah, et autres
Publié: (2026)

Stabilizing RNN Gradients through Pre-training
par: Herranz-Celotti, Luca, et autres
Publié: (2023)